DataStation
Install the App
Learn Blog Community Try the online demoarrow_forward
Blog

Writing an efficient object previewer for JavaScript

Published on July 15, 2021 by Phil Eaton
javascriptjson

In an earlier version of DataStation, previews of panel results were converted to JSON and then sliced to the first 100 characters. The point of these previews was to give the user feedback about the result of running a panel.

JSON.stringify()-based preview
JSON.stringify()-based preview

The code looked like this: 

function previewObject(obj: any, length: number): string {
  let str = JSON.stringify(obj, null, 2);
  if (str.length > length) {
    str = str.slice(0, length) + '...';
  }
  return str;
}

Performance implications

This was an easy enough solution and it worked alright at first. But as soon as you tried to preview large objects (at least a megabyte or so), the entire app would hang while calculating the preview.

This makes sense. It's a huge waste to convert the entire object to a JSON string just to show the first N characters.

My first instinct was to find a custom JSON stringifier library that would allow me to control the recursion depth. But when I couldn't find any, I started think about what "controlling the recursion depth" could mean.

The most obvious solution I could think of was showing a decreasing number of keys and array entries as you recurse through an object. The first version along these lines looked like this: 

function previewObject(obj, limit = 10) {
  if (!obj) {                                                            
    return String(obj);
  }           

  // Go down to zero at some point                                       
  const nextLimit = (limit / 2 < 1) ? 0 : (limit / 2);
  if (!nextLimit) {
    return "";
  }

  if (Array.isArray(obj)) {
    const previewedChildren = obj.slice(0, limit).map(c => previewObject(c, nextLimit));
    return '[ ' + previewedChildren.join(', ') + ' ]';
  }

  if (typeof obj === 'object') {
    const keys = Object.keys(obj).slice(0, limit);
    const previewedChildren = keys.map(k => '"' + k + '": ' + previewObject(obj[k], nextLimit));
    return '{ ' + previewedChildren.join(', ') + ' }';
  }

  let str = obj;
  if (typeof obj === 'string') {
    str = '"' + str + '"';
  }

  return String(obj).slice(0, 200);
}

Some key aspects of this are that we use String(thing) rather than thing.toString() so that if thing is ever null or undefined it will be properly stringified (and not throw an exception).

We also need to check if obj is an array before checking if it is an object since all arrays are objects in JavaScript. And we don't care about looking at hasOwnProperty in this case because the objects we're previewing in DataStation are always simple data objects.

One nice property of this implementation (unlike JSON.stringify) is that this is circular-object safe. If you pass a circular object to JSON.stringify it will throw an exception. This will not, since the number of keys looked at when recurse always goes to zero.

Finally, the most important part is that we're severely limiting the number of keys we iterate over. If we had no limits this code would be much slower than JSON.stringify (which is surprisingly fast). But again, this preview only needs to show a small subset of the object being previewed. So showing only a few keys is a great tradeoff.

When I replaced the JSON.stringify().slice() implementation with the custom algorithm, there was an immediate performance improvement. DataStation could handle many tens of megabytes of data loaded into the app now.

Getting more user-friendly

After getting this custom preview working, I wanted to make it more friendly. I wanted to have key-value and array elements show up on a newline if the top-level object was an array or object. This would make better use of the horizontal space available in the preview box in DataStation.

I also wanted to show that keys, elements, and cutoff string values were modified if they indeed were. (It isn't very important that the output becomes not-JSON.)

Here is the current implementation (I won't say final since it's likely there'll be even more improvements as time goes on): 

function unsafePreviewArray(
  obj: any,
  nKeys: number,
  stringMax: number,
  nextNKeys: number,
  prefixChar: string,
  joinChar: string
) {
  const keys = obj.slice(0, nKeys);
  const childPreview = keys.map(
    (o: string) => prefixChar + unsafePreview(o, nextNKeys, stringMax)
  );
  if (obj.length > nKeys) {
    childPreview.push(prefixChar + '...');
  }
  return ['[', childPreview.join(',' + joinChar), ']'].join(joinChar);
}

function unsafePreviewObject(
  obj: any,
  nKeys: number,
  stringMax: number,
  nextNKeys: number,
  prefixChar: string,
  joinChar: string
) {
  const keys = Object.keys(obj);
  keys.sort();
  const firstKeys = keys.slice(0, nKeys);
  const preview: Array = [];
  firstKeys.forEach((k) => {
    const formattedKey = `"${k.replaceAll('"', '\\"')}"`;
    preview.push(
      prefixChar +
        formattedKey +
        ': ' +
        unsafePreview(obj[k], nextNKeys, stringMax)
    );
  });

  if (keys.length > nKeys) {
    preview.push(prefixChar + '...');
  }

  return ['{', preview.join(',' + joinChar), '}'].join(joinChar);
}

function unsafePreview(
  obj: any,
  nKeys: number,
  stringMax: number,
  topLevel = false
): string {
  if (!obj) {
    return String(obj);
  }

  // Decrease slightly slower than (nKeys / 2) each time
  const nextNKeys = nKeys < 1 ? 0 : Math.floor(nKeys * 0.6);
  const joinChar = topLevel ? '\n' : ' ';
  const prefixChar = topLevel ? '  ' : '';

  if (Array.isArray(obj)) {
    return unsafePreviewArray(
      obj,
      nKeys,
      stringMax,
      nextNKeys,
      prefixChar,
      joinChar
    );
  }

  if (typeof obj === 'object') {
    return unsafePreviewObject(
      obj,
      nKeys,
      stringMax,
      nextNKeys,
      prefixChar,
      joinChar
    );
  }

  let res = String(obj).slice(0, stringMax);
  if (String(obj).length > stringMax) {
    res += '...';
  }

  if (typeof obj === 'string' && !topLevel) {
    res = `"${res.replace('"', '\\"')}"`;
  }

  return res;
}

export function previewObject(obj: any, nKeys = 20, stringMax = 200): string {
  try {
    return unsafePreview(obj, nKeys, stringMax, true);
  } catch (e) {
    console.error(e);
    return String(obj).slice(0, stringMax);
  }
}

Which ends up producing previews that look like this:

More efficient preview
More efficient preview

Not too bad!

If you have a better solution or find a major issue, I'd love to hear about it!

Share

With questions, criticism or ideas, email or Tweet me.

Also, check out DataStation and dsq.