July 22, 2022: Help us out by filling out a short user survey!

DataStation is an open-source data IDE for developers. It allows you to easily build graphs and tables with data pulled from SQL databases, logging databases, metrics databases, HTTP servers, and all kinds of text and binary files. Need to join or munge data? Write embedded scripts as needed in Python, JavaScript, Ruby, R, or Julia. All in one application.

DataStation 0.9.0 Release Notes

Published on

Happy April! This release of DataStation brings an improved data storage layer, support for basic autocomplete, and a number of UX improvements and bug fixes. There's also a small preview of an upcoming Enterprise Edition.

Check out this short demo of this release that connects to a CockroachDB instance, runs a basic query, and loads the results into a Pandas dataframe in Python for some exploration.

This video is also available as a tutorial.

Improved storage layer

The name sounds boring but this is actually a really big deal. Before this release all project state was stored in a single JSON blob that was sent on every request from the UI (with the exception of encrypted passwords).

Anyone familiar with API design recognizes this as a horrible model. It helped DataStation get this far but has too many chances for bugs and poor performance. In fact a number of bugs had been reported that were directly attribute to this storage method.

This release moves to an API-driven storage model where only individual entities are sent back on edit (e.g. the panel, connector, ssh server being edited). Additionally it moves from JSON to storing project data in a local SQLite file.

The good news is that the migration is done and well covered by automatic and manual testing. The bad news is that any migration like this can have bugs. DataStation automatically copies your existing JSON project file to a new file with a .bak suffix so you can recover your old data if the migration fails.

If you see bugs when running this new version please report them on Github or Discord! Although since this is a one-time migration the main recourse will be to help you get a valid SQLite version of your existing JSON project file.

Updated navigation

This release rearranges navigation slightly, adding a global sidebar to switch between the editor, dashboard, exports, and settings views. This has an important ramification for dashboards and exports; mainly that both will no longer be tied to a specific page in a project. When you use the server mode (which is still in testing) you will be able to put panels onto a dashboard from any page in the project.

Additionally, this new navigation makes it easier to access settings in desktop mode. And when you make changes in settings you'll no longer need to restart DataStation for the changes to take effect.

DataStation settings

Autocomplete

This release introduces very basic support for autocomplete. Autocomplete is now provided for panel names (for use when referencing panels with `DM_getPanel`) and panel result field names. Autocomplete is not yet provided for table names and columns when querying a database. The goal is definitely to do so eventually.

It's possible there are bugs in the autocomplete code that prevent you from editing anything at all. In that case you can disable autocomplete on the settings page.

DataStation autocomplete demo

Runner improvements

First-time contributor and first-time Go coder, Fritz, contributed two fixes to the DataStation runner code. The first was an improvement to DataStation's shape detection library. The second was to increase the size of lines the runner can handle when reading JSON newline files.

Thank you so much, Fritz!

Misc UI improvements

  • New panel creation requires you to pick a panel type, you can't edit panel type after creating a panel
    DataStation new panel
  • Stdout/Stderr section fixed (and tests added, finally); indicator appears when Stdout/Stderr is not empty
    DataStation stdout/stderr
  • Dark-mode is a little less blue/purple
    DataStation dark mode
  • Ctrl-tab and Ctrl-shift-tab switch between pages in desktop mode
  • Double clicking on a panel header toggles between panel fullscreen/regular size
  • New panels, database connectors, and ssh servers all start expanded when created

Enterprise Edition, Change History

As you build bigger workflows and reports with DataStation it becomes more important to be able to see the changes that you've made and be able to go backward. So the first paid-only feature is a full history of all changes made within DataStation.

This paid edition is not yet publicly available but if you're interested in being first to try it out, get in touch.

DataStation EE history

Install or upgrade

Get the 0.9.0 release now!

How you can help

Subscribe to updates!

If you are a developer or engineering manager, install DataStation and start using it at work! Report bugs and usability issues (there are surely many).

If you're a developer with some Go experience wanting to contribute to OSS, check out the Good First Projects page.

Share