A collection of technical notebooks from my books and publications, presented in chronological order.
Dominance relations can be clearly visualised when working in a two-objective space. Let's do this with some arbitrary solutions. We'll use the `ParetoFront` visualisation from PlotAPI.
The nodes in a PlotAPI Sankey diagram can be sorted (e.g. alphabetically) within their columns.
Everything you need to create beautiful, engaging, and interactive Goal Rush visualizations.
The popup supports displaying additional details and thumbnails. This makes exploration even better with PlotAPI Chord.
Frequently asked questions and their answers.
BarFight allows you to scroll up and down the number of bars, and set a starting point.
Everything you need to create beautiful, engaging, and interactive Split Chord visualizations.
Everything you need to create beautiful, engaging, and interactive Chord visualizations.
PlotAPI Chord supports pulling out arcs as a way to highlight or group elements.
Explore the different demonstrations of PlotAPI in this showcase.
We may want to switch between the WebGL or D3.js canvas animations depending on our requirements. These have been configured to look the same.
Let's take a look at the vertical property of the Terminus diagram. Using it can give the impression that the particles are "falling" through the pipes.
Everything you need to create beautiful, engaging, and interactive Terminus visualizations.
The stats panel that appears in a default Terminus diagram can be adjusted or hidden entirely. This gives us even more control over the presentation of our Terminus diagram.
Let's look at a unit-based approach to handling data with very high values or many sources and targets.
Let's look at a percentage-based approach to handling data with very high values or many sources and targets.
The Terminus pixel size can be modified to improve the presentation of our diagram, this works as intended in both WebGL and D3.js canvas mode.
The pixel color scheme can be set to one of many beautiful presets, or even a customized by supplying a list of colors (HEX, RGB, etc.).
The Terminus pipes can be modified to improve the presentation of the diagram, giving control over the spacing and slope positions.
The pipe color and opacity can be changed to improve the presentation of our Terminus diagram.
Let's take a look at the bundling parameter. This will group together the input, giving the impression that they are entering through the same pipe.
Pipes are aligned to the top-left by default, however, changing their alignment can often significantly improve the presentation of a Terminus diagram.
The title, width, height, margin, and position can be changed with these layout properties.
The Terminus diagram supports setting the title and also including the dynamic count of pixels currently on their journey.
The bars that appear at each terminus can be modified or hidden entirely. This gives us even more control over the presentation of our Terminus diagram.
The Terminus animation can be modified to improve the presentation of our diagram, giving us control over a pixel's journey duration, how many pixels are dispatched at once, and the delay before the animation begins.
The many text formatting options include multi-line label wrapping, size, and color, and much more.
Everything you need to create beautiful, engaging, and interactive Sankey visualizations.
Depending on our data, it may make more sense to reverse the gradient directions used for colouring the links.
The popup supports many customizations, including the text format, width, and disabling it entirely.
The nodes in a PlotAPI Sankey diagram can be styled with many different parameters.
The color scheme can be set to one of many beautiful presets, or even a customized by supplying a list of colors (HEX, RGB, etc.). Overrides also allow colouring specific nodes.
PlotAPI Sankey supports different layouts using the specification of node alignment.
PlotAPI Chord supports a linked data table. This means as you hover over arcs and chords in the Chord diagram, a data table will be filtering in real-time to show more information.
The opacity of the links in the Sankey diagram can be changed. The opacity will change on mouse-over to highlight the selected connections.
Until we mouseover a Sankey link, what we actually see is the background color. By default, this background colour is set to be the same as the link foreground color, as defined by the `colors` parameter, which by default is `"gradient"`.
The title, width, height, margin, position, background color, and border can be changed with these layout properties.
PlotAPI Sankey uses gradients to colour the links by default. This can be changed to use flat colors instead.
PlotAPI Sankey supports animations - both for interactions and for nice introductions to your visualisation.
Temporal formats can be used for order values and presentation.
Everything you need to create beautiful, engaging, and interactive Pie Chart Race visualizations.
Let's take a look at how we can change the colours and icons for nodes in our Pie Fight diagram. These configurations override the color scheme.
Let's take a look at how we can change the presentation of the text and values in our Pie Fight diagram.
The color scheme can be set to one of many beautiful presets, or even a customized by supplying a list of colors (HEX, RGB, etc.).
Event information can be displayed using different modes, these include paused, running, and interactive.
Let's take a look at how we can display event content during our visualisation at different times. This can be useful for displaying additional information or images that are relevant to specific events.
Everything you need to create beautiful, engaging, and interactive Pareto Front visualizations.
Temporal formats can be used for order values and presentation.
Let's take a look at how we can change the colours and icons for nodes in our Line Fight diagram. These configurations override the color scheme.
The color scheme can be set to one of many beautiful presets, or even a customized by supplying a list of colors (HEX, RGB, etc.).
Everything you need to create beautiful, engaging, and interactive Line Chart Race visualizations.
Event information can be displayed using different modes, these include paused, running, and interactive.
Let's take a look at how we can display event content during our visualisation at different times. This can be useful for displaying additional information or images that are relevant to specific events.
The popup supports many customizations, including the text format, width, and disabling it entirely.
Everything you need to create beautiful, engaging, and interactive Heat Map visualizations.
PlotAPI Chord supports displaying and formatting ticks around the diagram.
The many text formatting options include multi-line label wrapping, size, and color.
Depending on our data, it may make more sense to reverse the gradient directions used for colouring the ribbons.
The inner and outer radius scales of the Chord diagram can be adjusted. This will change the thickness of the arcs.
The popup supports many customizations, including the text format, width, and disabling it entirely.
PlotAPI Chord supports a linked data table. This means as you hover over arcs and chords in the Chord diagram, a data table will be filtering in real-time to show more information.
The title, width, margin, position, and rotation can be changed with these layout properties.
The arcs of a Chord diagram can be customised to be of equal size, regardless of the value of their relationships.
The default behaviour of PlotAPI chord is to represent both sides of a relationship with a single chord. However, it may be more suitable to use two different chords with arrows to indicate the dimension of a relationship.
In some cases, enabling curved labels will improve the look of a PlotAPI Chord diagram.
By default, Chord displays occurrences where a category is not related to another category. These values appear in the diagonal of the matrix. It may be desirable to hide (but not remove) these values.
The opacity of the inner section of the Chord diagram, or the _chords_, can be changed. The opacity will change on `mouseover` to highlight the selected connections.
The color scheme can be set to one of many beautiful presets, or even a customized by supplying a list of colors (HEX, RGB, etc.).
By turning symmetric mode off, each end of a chord can be of different size, and it also changes the information presented in the popup.
Adjusting padding may not sound exciting, but in this case, it can have some interesting effects.
PlotAPI Chord supports displaying the quantity associated with each arc as a label.
PlotAPI Chord supports animations - both for looping and for nice introductions to your visualisation.
Temporal formats can be used for order values and presentation.
Let's take a look at how we can change the number of bars that are visible by default.
Let's take a look at how we can change the colours and icons for nodes in our Bar Fight diagram. These configurations override the color scheme.
The color scheme can be set to one of many beautiful presets, or even a customized by supplying a list of colors (HEX, RGB, etc.).
The title, width, height, margin, position, background color, and border can be changed with these layout properties.
Event information can be displayed using different modes, these include paused, running, and interactive.
Let's take a look at how we can display event content during our visualisation at different times. This can be useful for displaying additional information or images that are relevant to specific events.
Everything you need to create beautiful, engaging, and interactive Bar Chart Race visualizations.
The BarFight animation can be modified to improve the presentation of our diagram, giving us control over animations durations, delays, and behavior.
Easily turn your data into engaging visualizations with PlotAPI's friendly interface — with or without code.
Welcome to the PlotAPI documentation. It contains all the information you need to get started using PlotAPI.
PlotAPI supports uploading visualizations to the cloud, where they can be embedded and shared privately, publicly, or within your Team.
PlotAPI supports saving visualizations locally. This includes saving to PNG, PDF, SVG, animated MP4, and interactive HTML.
PlotAPI supports more than just Python and Rust. With the REST API, you can create beautiful visualizations from any language.
How to use PlotAPI with Julia.
How to use PlotAPI with JavaScript.
How to use PlotAPI with HTTP.
Activate your license and get access to Pro and Business features with the instructions below.
Get access to the API with the instructions below. Libraries available for Python and Rust. For everything else, there's the REST API.
PlotAPI supports displaying interactive visualizations directly within Jupyter Notebooks. This includes Jupyter Lab, Jupyter Notebook Classic, Google Colab, and more.
In this notebook we're going to use PlotAPI Pie Fight to visualise desktop browser market share over time. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Bar Fight to visualise Pokémon search trends over time. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Chord to visualise the co-occurrences between Pokémon types. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Chord to visualise the co-occurrences between the species and personality of Animal Crossing villagers. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Sankey to visualise some of the Apple's filings for the third quarter of 2021.
In this notebook we're going to use PlotAPI Sankey to visualise some of the Apple's filings for the fourth quarter of 2021.
In this notebook we're going to use PlotAPI Chord to visualise the co-occurrences between League of Legends classes. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Heat Map to visualise the co-occurrences between Pokémon types. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Chord to visualise the style co-occurrences of Animal Crossing villagers. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Chord to visualise the co-occurrences of genres in the IMDb "Top 1000" (Sorted by IMDb Rating Descending). We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Terminus to visualise how degree classes vary by graduates' gender. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Terminus to visualise how degree classes vary by graduates' ethnicity. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Terminus to visualise the average daily email & spam volume for August 2021.
Data often needs wrangling prior to visualisation. Let's take a look at how we can transform our data from a DataFrame to a Dictionary for PlotAPI.
Choosing the right colours can often make all the difference when adding finishing touches to a visualization. Finding colours that work together can be difficult, even more so when we need to find colours for hundreds of elements. So let's see how we can generate some colours with code!
Data analysis and visualization tutorials often begin with loading an existing dataset. Sometimes, especially when the dataset isn't shared, it can feel like being taught how to draw an owl. So let's take a step back and think about how we may create a dataset for ourselves.
Whilst you could say that it's possible to draw a zigzag using multiple rect elements at different positions and rotations, it is certainly an infeasible and inefficient exercise. This is where the SVG path element comes in. A path describes an outline of some shape that can be filled and/or stroked.
A comparison between CVSS-2 and CVSS-3 using Python and the data science stack.
Chord Pro Features For Chord Diagrams
Whilst we could use the string literal magenta as our argument to change the fill colour, we'll use its hexadecimal equivalent, #ff00ff instead. This will give us more opportunity should we want to tinker with the colours. We could also use an RGB value, e.g. rgb(255, 0, 255).
D3.js transitions support easing functions which can change the speed at which an animation progresses throughout its duration. There are many different easing functions available in D3.js. Examples, descriptions, and visualisations of each one can be found in the API reference.
We've introduced this new invocation of the transition.on(typenames, listener) function. This can add a listener function to the selection, which is invoked based on the event type. We've used an event type of end because we want our transition to end before calling the next one.
Arabica Coffee Beans - Origin and Variety
Much like selections, transitions can be used to modify attributes and styles. The difference is that whilst selections apply the changes instantly, transitions apply the changes gradually (and smoothly) over a specified duration.
The g SVG element is a container used to group other SVG elements. Transformations applied to the g element are performed on its child elements, and its attributes are inherited by its children. We can create a group element with D3.js by appending a g element using any selection.
We can set CSS style properties by invoking .style(name, value) on the selection, and set SVG attributes by invoking .attr(name, value) where the argument to the first parameter should be the name of the attribute we want to set, and the argument to the second parameter should be the value we want to set it to.
Besides the d3.create() and d3.append() functions which return selections, we can use the d3.select() and d3.selectAll() functions to return selections by matching a CSS selector.
To create a circle element with D3.js we can invoke the append(name) function on our svg selection and pass in the name of the element. In this case, we're passing in circle as our argument for the name parameter.
To create an element with D3.js we invoke the create(name) function and pass in the name of the element. In this case, we're passing in svg as our argument for the name parameter.
It helps to have the right tools and templates available so that we can focus on the examples and exercises. Getting the hang of D3.js may involve plenty of tinkering, saving, and refreshing of HTML documents - don't be discouraged if your visualisations don't look right the first, second, or third time!
There is a wealth of cookbook-style resources available for D3.js visualisations, meaning you can create some interesting visualisations by copying some code and passing in your data. However, what this book aims to be is a practical journey through the many components of D3.js. By the end of this book, we want to be able to create new visualisations from the ground up and modify the behaviour of existing ones.
We work towards illustrating the age and height of athletes grouped by games in the 120 years of Olympic history.
Generate visualisations through the API from Python, Rust, and more. Super-charge your notebooks with inline visualisations!
We're going to use 120 years of Olympic history to create a visualisation. Let's set our sights on something that illustrates the distribution of Olympic medals awarded for the weightlifting sport.
In this notebook we're going to use PlotAPI SplitChord to visualise the co-occurrences between genres and publishers in video game titles. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Chord to visualise the co-occurrences between countries and medals earned in the olympic games. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Chord to visualise the matches between teams throughout the Legends World Championship 2019. We"ll use Python, but PlotAPI can be used from any programming language.
In this notebook we're going to use PlotAPI Chord to visualise the co-occurrences between panels and colours purchased for the StamiStudios Everyday Ita Bag. We"ll use Python, but PlotAPI can be used from any programming language.
We're going to use the Complete Pokemon Dataset dataset to visualise the co-occurrence of Pokémon types from generations one to eight. We'll make this happen using a chord diagram.
Co-occurrence of Anime Genres with Chord Diagrams
Co-occurrence of Movie Genres with Chord Diagrams
Chord diagrams are useful when trying to convey relationships between different entities, and they can be beautiful and eye-catching.
Co-occurrence of Pokemon Types (Gen 1-8) with Chord Diagrams
Occurrence and Co-occurrence of Pokemon Types with Chord Diagrams
Co-occurrence of Pokemon Types (Gen 1-6) with Chord Diagrams
Interactive Chord Diagrams
NumPy has many features that Rust's NDArray doesn't have yet, e.g. index arrays and mask index arrays. However, there is more than one way to index an array!
Let's use a bubble plot on a map of the UK to plot the confirmed cases of Coronavirus in the UK.
Coronavirus Time Series Line and Bar Chart
Let's demonstrate a few approaches to identifying the unique elements in an array, counting the number of unique elements, and the frequency of these unique elements.
Coronavirus Time Series Map Animation
We'll often need to make HTTP requests to retrieve data - let's see one common approach in Python.
Software Setup
Preface
We're going to take a look at some of the tools we have for descriptive statistics.
Let's demonstrate how to get parts of our raw string array into multiple arrays of various type.
We shift our focus to dealing with real-world datasets and how to load them into an NDArray.
Let's improve the presentation of the cell output for our arrays. This will generally improve the presentation of our notebooks.
The ndarray crate provides us with a multidimensional container that can contain general or numerical elements.
Let's take our Plotly workaround a step further to its final destination - a reusable function that we can use throughout our analyses.
Let's improve our workaround for data visualisation with Plotly for Rust in Jupyter notebooks.
How to embed Plotly visualisations in a Jupyter Notebook with a small workaround.
I had originally planned to use Plotters for all the graphing in this book. However, shortly after finding Plotters, I found out that a Rust library had enabled Plotly support.
We are taking a practical approach in the following sections. As such, we need the right tools and environments available in order to keep up with the examples and exercises.
The Rust programming language has become a popular choice amongst software engineers since its release in 2010. Besides being something new and interesting, Rust promised to offer exceptional performance and reliability.
What this section is most interested in is using YAML for configuration files, enabling us to extract parameters that we use within our programs so that they can be separated.
Let's use the Platypus framework to apply the Non-dominated Sorting Genetic Algorithm II (NSGA-II)1 to a custom objective function.
MNE DataFrame scaling with Pandas.
Plotting multiple channels in the time domain with Plotly.
Plotting multiple channels in the time domain.
Plotting event related potentials with MNE.
Plotting topographical maps with MNE.
How to plot multiple different sine waves onto different subplots.
Let's test the significance of our pairwise comparison. The significance test you select depends on the nature of your data-set and other criteria. We will use the Wilcoxon signed-rank.
We move on from creating and plotting individual sine waves, to summing them and plotting our new and more complicated wave.
How to plot multiple different sine waves onto different subplots.
Let's look at a simple sine wave, how to create one in Python and how to visualise it in the time domain using a line chart.
Before conducting a comparison between algorithms we need to determine whether our sample size will be sufficient, i.e. is our sample size large enough to support our hypothesis?
Let's use the Platypus framework to compare the performance of the Non-dominated Sorting Genetic Algorithm II (NSGA-II) and the Pareto Archived Evolution Strategy (PAES).
Let's demonstrate how we can use a popular multi-objective optimisation algorithm, NSGA-II, to approximate multiple trade-off solutions to the DTLZ2 test problem.
The Contributing Hypervolume (CHV) indicator is a population sorting mechanism based on an adaptation of the hypervolume indicator.
The hypervolume indicator is a performance metric for indicating the quality of a non-dominated approximation set.
Non-dominated sorting is important during the selection stage of an Evolutionary Algorithm because it allows us to prioritise the selection of solutions based on their dominance relations with respect to the rest of the population.
We often look for a single solution which has the best objective value, whereas this is not possible in multi-objective problems because they often involve conflicts between multiple objectives.
In single-objective problems, the objective is to find a single solution which represents the global optimum in the entire search space. Let's take the Rastrigin function as an example.
In single-objective problems, the objective is to find a single solution which represents the global optimum in the entire search space. Let's take the Sphere function as an example.
Let's use the Platypus implementation of ZDT1, which will save us from having to implement it in Python ourselves.
Let's demonstate how to generate and visualise a population in the objective space and decision space.
Before the main optimisation process can begin, we need to complete the initialisation stage of the algorithm. There are many schemes for generating the initial population - let's start simple.
We will be using a synthetic test problem throughout this notebook called ZDT2. It is part of the ZDT test suite, consisting of six different two-objective synthetic test problems.
We will be using a synthetic test problem throughout this notebook called ZDT1. It is part of the ZDT test suite, consisting of six different two-objective synthetic test problems.
Objective functions are perhaps the most important part of any Evolutionary Algorithm, whilst simultaneously being the least important part too.
Throughout this book, we will programmatically generate block diagrams to illustrate concepts and processes. Let's see how they're generated.
This crash-course makes the assumption that you already have some programming experience, but perhaps none with Python.
We are taking a practical approach in the following sections. As such, we need the right tools and environments available in order to keep up with the examples and exercises.
Evolutionary Algorithms (EAs) are a fascinating class of algorithms for meta-heuristic optimisation. Perhaps the most difficult question to answer is where do we start? There is so much to cover, and many potential starting points.
We use adaptive synthetic sampling (ADASYN) to resample and balance our Cardiotocography dataset.
Let's take a quick look at the problem of imbalanced datasets and one way to address it with oversampling.
Supported by figures and statistics, we will have a look at how our solution performed and discuss anything interesting about the results.
We will use the Keras API on top of TensorFlow to implement our experiment. All code will be in Python, and at the time of publishing everything is guaranteed to work within a Kaggle Notebook.
It's important to know what we're looking for, how we're going to use our dataset, what algorithms we will be employing, and how we will determine whether the performance of our approach is successful.
We present and discuss a dataset selected for our machine learning experiment. This includes some analysis and visualisations to give us a better understanding of what we're dealing with.
Let's go through the process of signing up to Kaggle, firing up a Kernel, and executing a Hello World program in Python.
Pairwise comparison of data-sets is very important. It allows us to compare two sets of data and make decisions based on the outcome.
A brief re-cap on calculating the standard deviation with and without numpy.
We are taking a practical approach in the following sections. As such, we need the right tools and environments available in order to keep up with the examples and exercises.