Happy April! This release of DataStation brings an improved data storage layer, support for basic autocomplete, and a number of UX improvements and bug fixes. There's also a small preview of an upcoming Enterprise Edition.

Check out this short demo of this release that connects to a CockroachDB instance, runs a basic query, and loads the results into a Pandas dataframe in Python for some exploration.

This video is also available as a tutorial.

Improved storage layer

The name sounds boring but this is actually a really big deal. Before this release all project state was stored in a single JSON blob that was sent on every request from the UI (with the exception of encrypted passwords).

Anyone familiar with API design recognizes this as a horrible model. It helped DataStation get this far but has too many chances for bugs and poor performance. In fact a number of bugs had been reported that were directly attribute to this storage method.

This release moves to an API-driven storage model where only individual entities are sent back on edit (e.g. the panel, connector, ssh server being edited). Additionally it moves from JSON to storing project data in a local SQLite file.

The good news is that the migration is done and well covered by automatic and manual testing. The bad news is that any migration like this can have bugs. DataStation automatically copies your existing JSON project file to a new file with a .bak suffix so you can recover your old data if the migration fails.

If you see bugs when running this new version please report them on Github or Discord! Although since this is a one-time migration the main recourse will be to help you get a valid SQLite version of your existing JSON project file.

Updated navigation

This release rearranges navigation slightly, adding a global sidebar to switch between the editor, dashboard, exports, and settings views. This has an important ramification for dashboards and exports; mainly that both will no longer be tied to a specific page in a project. When you use the server mode (which is still in testing) you will be able to put panels onto a dashboard from any page in the project.

Additionally, this new navigation makes it easier to access settings in desktop mode. And when you make changes in settings you'll no longer need to restart DataStation for the changes to take effect.

Autocomplete

This release introduces very basic support for autocomplete. Autocomplete is now provided for panel names (for use when referencing panels with `DM_getPanel`) and panel result field names. Autocomplete is not yet provided for table names and columns when querying a database. The goal is definitely to do so eventually.

It's possible there are bugs in the autocomplete code that prevent you from editing anything at all. In that case you can disable autocomplete on the settings page.

Runner improvements

First-time contributor and first-time Go coder, Fritz, contributed two fixes to the DataStation runner code. The first was an improvement to DataStation's shape detection library. The second was to increase the size of lines the runner can handle when reading JSON newline files.

Thank you so much, Fritz!

Misc UI improvements

• New panel creation requires you to pick a panel type, you can't edit panel type after creating a panel
  
• Stdout/Stderr section fixed (and tests added, finally); indicator appears when Stdout/Stderr is not empty
  
• Dark-mode is a little less blue/purple
  
• Ctrl-tab and Ctrl-shift-tab switch between pages in desktop mode
• Double clicking on a panel header toggles between panel fullscreen/regular size
• New panels, database connectors, and ssh servers all start expanded when created

Enterprise Edition, Change History

As you build bigger workflows and reports with DataStation it becomes more important to be able to see the changes that you've made and be able to go backward. So the first paid-only feature is a full history of all changes made within DataStation.

This paid edition is not yet publicly available but if you're interested in being first to try it out, get in touch.

Install or upgrade

Get the 0.9.0 release now!

How you can help

Subscribe to updates!

If you are a developer or engineering manager, install DataStation and start using it at work! Report bugs and usability issues (there are surely many).

If you're a developer with some Go experience wanting to contribute to OSS, check out the Good First Projects page.

A new DataStation release is out! This one brings support for basic autocomplete of panel names and result fields, a much-improved storage layer, and a number of other UX and runner improvements!

And a first look at an upcoming paid Enterprise Edition 🎉https://t.co/JF86MDlNs4 pic.twitter.com/6cyj8Rx7nN

— Multiprocess Labs (@multiprocessio) April 8, 2022