DataStation 0.4.0 Release Notes

What is DataStation?DataStation is an open-source data IDE for developers. It allows you to easily build graphs and tables with data pulled from SQL databases, logging databases, metrics databases, HTTP servers, and all kinds of text and binary files. Need to join or munge data? Write embedded scripts as needed in Python, JavaScript, Ruby, R, or Julia. All in one application.

DataStation 0.4.0 Release Notes

Published on December 6, 2021

UI Updates (and dark mode)

The primary change in this release is an updated UI and a dark mode. You can try out both in in the online demo.

The desktop app dark mode setting is turned on by editing the JSON file `~/DataStationProjects/.settings`, adding `"theme": "dark"` to the top level, and restarting DataStation.

Confirmed processing files up to 512mb

DataStation can now definitely process individual files up to 512mb in size and likely much larger. Exactly how much larger has not yet been tested.

There were two limiting factors here. One was the size of a single string in Node.js. This was fixed by writing array data out to files element-at-a-time rather than JSON.stringify-ing the entire array once. The other limiting factor was discovering the shape of large files. This was fixed by building a partial-JSON parser that stops reading a file after a certain number of bytes and closes all open arrays/objects to create a valid JSON object.

Install or upgrade

Get the 0.4.0 release now!

How you can help

If you are a developer or engineering manager, install DataStation and start using it at work! Report bugs and usability issues (there are surely many). Get in touch if you are interested in being a design partner.

If you are an investor, get in touch.