DevKit

The OpenSnowcat DevKit provides an easy local development environment using docker containers.

https://github.com/opensnowcat/opensnowcat-devkit

DevKit uses the following components:

• Kafka / Warpstream with Kafka UI for observability
• OpenSnowcat Collector Kafka
• OpenSnowcat Enricher Kafka

Kafka

Start the environment with:

docker compose up -d

Open your browser and visit KafkaUI: https://localhost:8081 to inspect your Kafka topics and messages. Topics are created automatically depending on the collector and enricher configurations.

WarpStream

WarpStream is an Apache Kafka® compatible data streaming platform built directly on top of object storage. Open your browser and visit KafkaUI: https://localhost:8081 to inspect your Kafka topics and messages.

Look at Wapstream container log to find the Warpstream control plane URL.

docker compose -f docker-warpstream.yml up

When using Warstream also open the Warpstream control plane (see URL in the Warpstream docker container log)

Sending events to the collector

The easiest way to send events to the collector is using ngrok to map your local collector to a public https endpoint and then use Snowplow Chrome Debugger collector override feature.

Open a terminal and run ngrok.

ngrok http 8080

Use the ngrok URL https://xxxxxxxx.ngrok-free.app in the SnowcatCloud Chrome Extension in the collector override.