DataStationDataStation is an open-source data IDE for developers. It allows you to easily build graphs and tables with data pulled from SQL databases, logging databases, metrics databases, HTTP servers, and all kinds of text and binary files. Need to join or munge data? Write embedded scripts as needed in Python, JavaScript, Ruby, R, or Julia. All in one application.
DataStation 0.4.0 Release Notes
UI Updates (and dark mode)
The primary change in this release is an updated UI and a dark mode. You can try out both in in the online demo.
The desktop app dark mode setting is turned on by editing the JSON file `~/DataStationProjects/.settings`, adding `"theme": "dark"` to the top level, and restarting DataStation.
Confirmed processing files up to 512mb
DataStation can now definitely process individual files up to 512mb in size and likely much larger. Exactly how much larger has not yet been tested.
There were two limiting factors here. One was the size of a single
string in Node.js. This was fixed by writing array data out to files
element-at-a-time rather than JSON.stringify-ing the
entire array once. The other limiting factor was discovering the
shape of large files. This was fixed by building a partial-JSON
parser that stops reading a file after a certain number of bytes and
closes all open arrays/objects to create a valid JSON object.
Install or upgrade
How you can help
If you are a developer or engineering manager, install DataStation and start using it at work! Report bugs and usability issues (there are surely many). Get in touch if you are interested in being a design partner.
If you are an investor, get in touch.