Skip to content
You are reading Codefi Orchestrate development version documentation and some displayed features may not be available in the stable release. You can switch to stable version using the version box at screen bottom.

Connect Apache Kafka

Codefi Orchestrate uses Apache Kafka as message broker and streaming platform used to listen for transactions. Follow these steps to configure Kafka when starting the following services:

Tip

Configure each microservice using microservice-specific environment variables. Command line options are also available and take precedence over environment variables.

Global Configuration

Environment Variable Command line option Description Default
KAFKA_URL kafka-url List of URLs of Kafka servers to connect to localhost:9092
KAFKA_VERSION kafka-version Version of Kafka that Orchestrate will assume it is running against 1.0.0

SSL/TLS Configuration

In addition to the options above, follow these steps to configure Kafka over TLS communication.

Environment Variable Command line option Description Default
KAFKA_TLS_ENABLED kafka-tls-enabled Whether or not to use TLS false
KAFKA_TLS_INSECURE_SKIP_VERIFY kafka-tls-insecure-skip-verify Whether the client verifies the server’s certificate chain and host name false
KAFKA_TLS_CLIENT_CERT_FILE kafka-tls-client-cert-file TLS certificate file path
KAFKA_TLS_CLIENT_KEY_FILE kafka-tls-client-key-file TLS key file path
KAFKA_TLS_CA_CERT_FILE kafka-tls-ca-cert-file TLS Certificate Authority file path

When using TLS communication in production, it is not recommended to set kafka-tls-insecure-skip-verify to true as that will open a breach to man-in-the-middle attacks.

SASL Configuration

In addition to the options above, follow these steps to configure Kafka SASL authentication.

Environment Variable Command line option Description Default
KAFKA_SASL_ENABLED kafka-sasl-enabled Whether or not to use SASL authentication false
KAFKA_SASL_MECHANISM kafka-sasl-mechanism SASL mechanism (PLAIN, OAUTHBEARER) PLAIN
KAFKA_SASL_HANDSHAKE kafka-sasl-handshake Whether or not to send the Kafka SASL handshake first. You should only set this to false if you’re using a non-Kafka SASL proxy. true
KAFKA_SASL_USER kafka-sasl-user Username for SASL/PLAIN or SASL/SCRAM authentication
KAFKA_SASL_PASSWORD kafka-sasl-password Password for SASL/PLAIN or SASL/SCRAM authentication
KAFKA_SASL_SCRAMAUTHZID kafka-sasl-scramauthzid ID used for SASL/SCRAM authentication

Configuration: Orchestrate Gateway API

Specific Kafka Configuration options when starting Orchestrate Gateway API. You can use the default values if you are not using multiple instances of Orchestrate.

Environment Variable Command line option Description Default
TOPIC_TX_SENDER topic-tx-sender Topic for messages between the API and the Tx-Sender topic-tx-sender

Configuration: Orchestrate Transaction Sender

Specific Kafka Configuration options when starting Orchestrate Transaction Sender in addition to the global Kafka options.

Apache Kafka is certainly the most delicate of Orchestrate’s dependencies, we recommend using the default values unless there is a need to modify them for advance usage.

Environment Variable Command line option Description Default
KAFKA_CONSUMER_GROUP_NAME consumer-group-name Kafka consumer group name group-sender
TOPIC_TX_RECOVER topic-tx-recover Topic for failed transaction messages topic-tx-recover
TOPIC_TX_SENDER topic-tx-sender Topic for messages between the API and the Tx-Sender topic-tx-sender
KAFKA_CONSUMER_MAX_WAIT_TIME kafka-consumer-max-wait-time Maximum amount of time the broker will wait in ms 250ms
KAFKA_CONSUMER_MAXPROCESSINGTIME kafka-consumer-max-processing-time Maximum amount of time in ms the consumer expects a message to be processed 100ms
KAFKA_CONSUMER_GROUP_SESSION_TIMEOUT kafka-consumer-group-session-timeout Timeout in seconds used to detect consumer failures when using Kafka’s group management facility 10s
KAFKA_CONSUMER_GROUP_HEARTBEAT_INTERVAL kafka-consumer-group-heartbeat-interval Expected time in seconds between heartbeats to the consumer coordinator when using Kafka’s group management facilities 3s
KAFKA_CONSUMER_GROUP_HEARTBEAT_INTERVAL kafka-consumer-group-heartbeat-interval Expected time between heartbeats to the consumer coordinator when using Kafka’s group management facilities 3s
KAFKA_CONSUMER_GROUP_REBALANCE_TIMEOUT kafka-consumer-group-rebalance-timeout Maximum allowed time in seconds for each worker to join the group once a re-balance has started 60s
KAFKA_CONSUMER_GROUP_REBALANCE_STRATEGY kafka-consumer-group-rebalance-strategy Strategy for allocating topic partitions to members (Range, RoundRobin, Sticky) Range

Configuration: Orchestrate Transaction Listener

Specific Kafka Configuration options when starting Orchestrate Transaction Listener in addition to the global Kafka options. You can use the default values if you are not using multiple instances of Orchestrate.

Environment Variable Command line option Description Default
TOPIC_TX_DECODED topic-tx-decoded Topic for successful transaction messages (receipts) topic-tx-decoded
ConsenSys has acquired Quorum from J.P. Morgan. Please read the FAQ.
Questions or feedback? You can obtain paid professional support by Consensys at orchestrate@consensys.net