Pulsar connectors enable Pulsar to quickly and easily integrate with various external systems. In fact, according to the 2021 Pulsar User Survey Report (which will be published later this month), connectors are one of the most-used Pulsar features with 30% of Pulsar users using connectors.
To facilitate connector development and improve their ease of use, we launched StreamNative Hub in 2020 to provide a single place to find, download, use, store, and share Pulsar related extensions, and offer a broad spectrum of Pulsar integrations. Since its launch last year, dozens of connectors have been created and added to the Hub. Some popular Pulsar plugins on StreamNative Hub include AWS SQS connector, AWS Lambda connector, AMQP1_0 connector, IoTDB connector, and more.
In this blog, we introduce recent updates that make developing and using Pulsar connectors even easier.
StreamNative Hub is an app store for developing event streaming applications and provides dozens of plugins and integrations. Its key components include:
As more and more members have contributed and used connectors, we’ve identified some opportunities to improve the Hub’s ease of use, read on to learn more.
To simplify the integration between Pulsar and external systems, we created a new development guide, Pulsar Connector Development Guide, that developers can reference to improve productivity and boost efficiency when developing a connector. This guide helps with the following:
Developing a New Connector
If you need to pipe data in or out of Pulsar and other systems that do not have a connector yet, you can read the Pulsar Connector Development Guide. It contains step-by-step guidelines for how to develop and contribute a connector to StreamNative Hub, including detailed instructions and various templates for both code and documentation.
Promoting Awareness and Usage of an Existing Connector
If you already developed a connector and want to make it available to the community, we recommend you host it in a public repository and show it on StreamNative Hub. You can host the connector repo at your desired location and then sync the documentation to StreamNative Hub using a simple script with just one line of code by following the instructions in the Pulsar Connector Development Guide.
We are continuously looking for new ways to improve StreamNative Hub and we are working on additional upgrades, such as adding more comprehensive tests to improve the usability, reliability, and performance of connectors. You can also expect more connectors to be deployed and adopted on more cloud providers with GUI tools. Stay tuned!
If you develop connectors, we encourage you to add your connector to StreamNative Hub! In StreamNative Hub, your connector will get exposure to the widest possible audience and enjoy faster innovation cycles of development. You will also be contributing to a robust Pulsar ecosystem.
In addition to adding a connector, there are more ways you can contribute, including:
The documentation hosted at StreamNative Hub is open source. Feel free to submit or request changes (fix typos, add clarifications, and more).
Start your journey with connectors now with the Quick Start Guide!