Google Vision API Is Not Working After Upload Image To Firebase

I built a image detection mobile app (e.g. Plastic Bottle, Aluminum Can, Milk Jug, etc.) with React-Native by using google vision API. It worked well before and got response succes

Solution 1:

I found the reason, but I am still curious why. Fetch blob and google vision seems conflict each other. I changed Firebase image upload function, and it worked well.

Following is my modified Firebase image upload function.

const imageUploadToFirebase = () => {
      var path = Platform.OS === 'ios' ? scannedURI.replace('file://', '') : scannedURI;
      const response = await fetch(path)
      const blob = await response.blob();
      var newItemKey = Firebase.database()
        .ref()
        .child('usersummary')
        .push().key;
      var _name = newItemKey + 'img.jpg';
      Firebase.storage()
        .ref(_name)
        .put(blob)
        .then(() => {
          return Firebase.storage()
            .ref(_name)
            .getDownloadURL();
        })
        .then(async uploadedFile => {
          let image = selectImage(sendItem.name?.toLowerCase());
          sendItem.image = image;
          sendItem.scannedURI = uploadedFile;
          AsyncStorage.getItem('@scanedItemList')
            .then(res => {
              if (res != null && res != undefined && res != '') {
                let result = `${res}#${JSON.stringify(sendItem)}`;
                AsyncStorage.setItem('@scanedItemList', result);
              } else {
                AsyncStorage.setItem(
                  '@scanedItemList',
                  JSON.stringify(sendItem),
                );
              }
            })
            .catch(err => console.log(err));
        })
        .catch(error => {
          console.log({error});
        });
}

Solution 2:

I'm not sure if you are using @google-cloud/vision package (in the callGoogleVisionApi() function) but as far as I know that is meant to be used in server side and authenticate with a service account. As an alternative to this method, you can use Cloud Storage Triggers for Cloud functions which will trigger a function whenever a new file is uploaded and then use Cloud Vision API.

Solution 3:

The Google Vision API can use a base64-encoded image, a publicly accessible HTTP URI, or a blob in google cloud storage.

In order to use an HTTP URI you should change the JSON payload from your callGoogleVisionAPI function from this:

{
        "requests": [{
          "image": { "content": base64 },
          features: [
            { type: "LABEL_DETECTION", maxResults: 30 },
            { type: "WEB_DETECTION", maxResults: 30 }
          ],
        }]
      }

to this:

{
        "requests": [{
          "image": { "source": {"imageUri": 'https://PUBLIC_URI_FOR_THE_IMAGE' }  },
          features: [
            { type: "LABEL_DETECTION", maxResults: 30 },
            { type: "WEB_DETECTION", maxResults: 30 }
          ],
        }]
      }

You've got a better explanation here: Make a Vision API request.