Image-scraper NodeJS. How To Send Callback Function To Result Array

I tried to build easy app to build img-parser and started to use library image-scraper(node-image-scraper). And faced with a problem. The question is: How could I get final array o

Solution 1:

If you want a promise, then scraper#scrape() can be promisified.

var Scraper = require("image-scraper");

Scraper.prototype.scrapeAsync = function(ms) {
    var ref = this; // same coding style as in existing methods.
    var images = [];
    return new Promise(function(resolve, reject) {
        ref.on('image', (image) => { images.push(image) });
        ref.on('end', () => { resolve(images) });
        // ref.on('error', reject); // unfortunately image-scraper doesn't emit an 'error' event.
        if(ms !== undefined) { // maybe timeout as substitute for error handler?
            setTimeout(() = {
                reject(`image-scraper timed out after ${ms} ms`);
            }, ms);
        }
        ref.scrape();
    });
}

untested

Call, eg :

const scraper = new Scraper('whatever');

scraper.scrapeAsync(30000).then((images) => {
    // process the `images` array here.
});

It should be reasonably simple to modify the image-scraper source to emit "error" events instead of logging errors. You will probably want separate events for page_error (fatal) and image-error (non-fatal).

There seems to be little point submitting a pull-request - last update was 2 years ago.

Solution 2:

Use the scraper.on method to listen for the end event.

Note that your call to .scrape(callback) can also be replaced with .on('image', callback).

var images = []

scraper.on('image', function (image) {
  images.push(image)
})

scraper.on('end', function () {
  console.log(images)
})