Beautiful interactive chord diagrams with Chord Pro

When nothing comes to mind, just make up your own quote.

This brief article tells the journey of my first software release with a price tag. Read on to find out how I launched my first SaaS with D3.js, FastAPI, and WooCommerce.

It all started when I wanted to write a section on Chord diagrams for my book, Data is Beautiful (if you've seen the book, you'll know that my relationship with chord diagrams got a little out of hand). The plan was to compile the book with Python notebooks and follow the literate programming paradigm. This meant I needed a Python package for Chord diagram generation.

I surveyed many Python packages for Chord diagram generation, these included Plotly, Bokeh, and a few lesser-known packages. I wanted something as beautiful as it was interactive, but I couldn't find one with the right balance. I had wanted to take a closer look at D3.js anyway, so I treated this as an opportunity to create my own Python package for Chord diagram visualisation!

Beautiful By Default

Chord Pro started with two core ideas. It was to be:

• Beautiful and interactive by default,
• and straight-forward to invoke, with many optional customisations.

With that in mind, I converged toward a single HTML file that rendered an interactive chord diagram. On this side of things, I stuck with HTML, CSS, JavaScript, and D3.js.

I was happy to promote this as "beautiful by default".

For tooltips, I was lucky enough to find and settle on Tippy.js. I was satisfied with its appearance with only minor configuration.

Customisations

I parameterised several customisations based on my personal requirements and feedback given by others along the way. These included the following four examples.

Multiple Colour Schemes

Bipartite Support

Rich Popups

Reversed Gradients

Creating and Exposing the API

For the server-side I wanted something lean and not too opinionated, i.e. I didn't want to be forced into a particular pattern. I was doing a lot of Python work back then and some options I found were Flask, Django, and FastAPI.

• Flask: I had some experience here exposing machine learning algorithms over the web. It looked like a good option.
• Django: I had only read about Django before, and after some more reading I decided I didn't want to learn the "Django way" of doing things, and the "batteries included" were not of much use for this project. Spoiler for future articles: I end up using Django to create PlotPanel.com.
• FastAPI: This was completely new to me and the articles/discussions I found were advocating FastAPI over Flask.

FastAPI was very convincing, and I ended up choosing it to build the Chord Pro service. To summarise, the API end-point would receive the data and parameters, process them, load them into the templated HTML file from earlier, and return the HTML as a string.

For the interested reader, this was hosted on AWS using docker containers and docker-compose.

Handling Payment and Licenses

Chord Pro was to use a single-payment model, so that made things a little easier. I didn't want to build and maintain a system for accepting payments and managing accounts/licenses, so I went with something I already understood and had available. I was already selling books in the DataCrayon Shop with WordPress + WooCommerce, so I added Chord Pro as a product and used the WordPress API to authenticate license holders.

I've read a lot of arguments for/against WordPress... Was it perfect? No! But in this instance it allowed me to get payments and accounts out the way fairly quickly. It also allowed me to accept many payment methods with its payment plugins.

Creating a Python Package

Whilst Chord Pro did technically support any environment that allowed for HTTP requests, I wanted to create a Python package to make things easier for notebooks. It was my first time doing this too, but it was easy enough to get something on PyPi that could be installed with pip install chord.

Getting things to behave in notebooks was a challenge. There were so many notebook environments that did things slightly or very differently. The crucial one was the difference between Jupyter Notebook (classic) and Jupyter Lab. I pushed onwards with support for only Jupyter Lab, as Jupyter were referring to it as the "next generation" notebook interface. Another spoiler for the future where I launch Plotapi.com: I end up supporting Jupyter Notebook, Jupyter Lab, Google Colab, and much more.

Creating a Rust Crate

I'll keep this part brief - but I also wrote a Rust Crate for Chord Pro users!

My Version of Success

I was very happy and flattered with what I consider to be the success of Chord Pro:

• It actually worked! I used it for my book and in notebooks for my own analyses.
• I had customers! The feedback was generally great, and people started using Chord Pro and sharing their visualisations with me and others. It was a one-time purchase and not very pricey (originally £5, slowly moving up to £9 as the features increased). The amount of money it generated was enough to support a coffee habit of 4 drinks a month. If I was to take my own time into account, the project lost money!
• I started posting visualisations on Reddit and three of them made it to the front page! This didn't result in a significant increase in purchases, but it did increase my AWS bills because of poor caching, and plenty of downtime because of a weak server!

Unexpected Challenges

I learned a lot from this project. Here are some things I wasn't prepared for that ultimately led to Chord Pro shutting down.

Free Users

Chord Pro used to have a free end-point with limited features. This went well for a while without needing much attention, but one night it exploded with usage and the server simply couldn't cope. I eventually found that someone had used it in their article on a popular Medium.com blog. It was great to see Chord Pro featured, but with the high came the low! Usage barely slowed down from that point onwards, perhaps because it was very convenient to install with pip and copy/paste an example followed by some tinkering.

I eventually had to shut down the free API end-point to return the paid user experience to acceptable. I tried a few things before that, e.g. rate-limiting, but my server setup at the time meant it wasn't feasible.

Users Breaking ToS

This one was tricky and I realised it too late. Chord Pro was advertised as a tool for generating visualisations that you could then host/distribute without restriction. It was stated in the FAQ and ToS that it wasn't to be used as part of a user-facing system, i.e. you couldn't use it to create a Chord Diagram generator on your website.

It's only after disabling the free end-point that I discovered one particular user had generated more visualisations than all the free and paid users put together! They had integrated Chord Pro into their dashboard which was serving many (thousands of) users a day. When I said I realised it too late, my AWS bill caught me off guard. My solution was to refund the user and disable their license.

End of Service

The above issue grew slowly over time, when I suspect users had finished integration of Chord Pro into some part of their app and gained some users. The AWS bills weren't as high as the previous case with the user I refunded/disabled, but they were consistently at a cost where Chord Pro had lost more than it had gained financially. I then realised many users weren't self-hosting their own scripts as mentioned in the manual, so I'd also become a CDN! It's likely most users hadn't read the manual, which is understandable, I don't read the manual for most things either.

I sent some emails out at this point, but I'm convinced many of them were sent straight to the spam inbox.

I put measures in place to prevent these issues for the next project I was working on, Plotapi.com, which was a subscription-based service for data visualization that included Chord diagrams amongst many others. I had a difference in attitude with PlotAPI - it was something I expected to dedicate significant time to rather than something that started as a hobby. Chord Pro was eventually shut down because of the costs, and users who did save their scripts can still view and distribute their generated visualisations forever!