D3 — Progressively Draw A Large Dataset

I'm using d3.js to plot the contents of an 80,000 row .tsv onto a chart. The problem I'm having is that since there is so much data, the page becomes unresponsive for aprox 5 secon

Solution 1:

I think you'll have to chunk your data and display it in groups using setInterval or setTimeout. This will give the UI some breathing room to jump in the middle.

The basic approach is: 1) chunk the data set 2) render each chunk separately 3) keep track of each rendered group

Here's an example:

var dataPool = chunkArray(data,100);
function updateVisualization() {
    group = canvas.append("g").selectAll("circle")
    .data(dataPool[poolPosition])
    .enter()
    .append("circle")
    /* ... presentation stuff .... */

}

iterator = setInterval(updateVisualization,100);

You can see a demo fiddle of this -- done before I had coffee -- here:

http://jsfiddle.net/thudfactor/R42uQ/

Note that I'm making a new group, with its own data join, for each array chunk. If you keep adding to the same data join over time ( data(oldData.concat(nextChunk) ), the entire data set still gets processed and compared even if you're only using the enter() selection, so it doesn't take long for things to start crawling.