Product Library search fails to authenticate with WORKING credentials

Hello all,

I’m relatively new to SNAP, and this is my first post to STEP (!), but I’ve been careful to reconfirm the issue that I’m having.

I can visit the Data Hub in a browser and successfully sign in. (I did change the password earlier today, but I have logged out and logged back in ( a few times) to confirm that the new credentials work and are properly kept in my password manager.

When I go into the Product Library settings, select Scientific Data Hub, and enter these (copy/paste from password manager, and visually inspect) into the credentials, click Apply, and attempt a search within SNAP, I get a failure alert window (“Failed to retrieve product list from Scientific Data Hub”) and the ~/.snap/var/log/messages.log file includes:

FINE [global]: HTTP GET https://scihub.copernicus.eu/apihub/search?rows=1&start=0&...  returned 401
SEVERE [org.esa.snap.product.library.ui.v2.thread.AbstractRunnable]: Failed to retrieve the product list from 'Scientific Data Hub'.
ro.cs.tao.datasource.QueryException: The supplied credentials are invalid!
	at ro.cs.tao.datasource.remote.scihub.SciHubDataQuery.getCount(SciHubDataQuery.java:228)

I’ve removed the bulk of the search URL above, but it isn’t the issue. However, I will add that, if I do manually copy/paste the full URL back into my browser, I am prompted in a small window for a user/password. And the “working” credentials seem to fail here too, as I am prompted again.

I have restarted SNAP, and reconfirmed the Data Hub account credentials in the settings and tried again to no avail.

I’m using the latest SNAP (8.0.3? “About SNAP” menu doesn’t give me any informational window) on MacOS 11.2.2

Any help is greatly appreciated!

Alaric

Your “working” credentials are not the one used in Product Library. The error message is quite clear. Maybe an extra character was pasted (an empty space, etc).
Check them in Tools -> Options -> General -> Product Library, Scientific Data Hub. On the right, there is an “eye” button to see the password in clear.

If the credentials used in the URL from Product Library also fail in the browser, it confirms that they are not the correct ones.

Try again your new credentials in
https://scihub.copernicus.eu/dhus/#/home
(you can log out and log in again with the new credentials to confirm them)

Thank you for the reply!

Yes, I was very careful to click the “eye” icon yesterday, and confirm the password had been stored correctly, several times actually, and even tried manually typing it in rather than pasting.

I tried again this morning, without changing anything, and things worked! I have a feeling that my new password needed to propagate to some other server for the SNAP Product Library query than the one used for a browser login…

I’m sorry to have bothered, but it was definitely a mysterious problem…

Thanks again!
Alaric

1 Like

No problem :slight_smile:, it’s good that it worked in the end.

HI

I encountered the same issue, however my credentials are working in scientific hub. I checked the correct functioning by using a different computer and asking a colleague to create a new user…with both solutions the same issue happened.

Also I checked downgrading to Version 7.0 and the product library search worked perfectly, so I can confirm that is a problem of Version 8.03 (both SNAP desktop implementation and engine implementation)

Here’s the output:

    ro.cs.tao.datasource.QueryException: The supplied credentials are invalid!
	at ro.cs.tao.datasource.remote.scihub.SciHubDataQuery.getCount(SciHubDataQuery.java:228)
	at org.esa.snap.remote.products.repository.download.RemoteRepositoriesManager.downloadProductList(RemoteRepositoriesManager.java:368)
	at org.esa.snap.remote.products.repository.download.RemoteProductsRepositoryProviderImpl.downloadProductList(RemoteProductsRepositoryProviderImpl.java:65)
	at org.esa.snap.product.library.ui.v2.repository.remote.download.DownloadProductListTimerRunnable.execute(DownloadProductListTimerRunnable.java:80)
	at org.esa.snap.product.library.ui.v2.repository.remote.download.DownloadProductListTimerRunnable.execute(DownloadProductListTimerRunnable.java:26)
[catch] at org.esa.snap.product.library.ui.v2.thread.AbstractRunnable.run(AbstractRunnable.java:39)
	at java.lang.Thread.run(Thread.java:748)

Thx !

Mario

Hello,
The error is misleading.
The cause is that SciHub API URL changed. They also announced this in their news:
https://scihub.copernicus.eu/news/News00868
The API URL used in SNAP Product Library should also be updated. I am creating a ticket for this:
SNAP-1435

1 Like

It seems that the redirect mentioned in the announcement does not work properly.
Should we contact them regarding this?

Yes, the redirect does not work properly, causing the misleading error that we get in SNAP.
The libraries used in SNAP for SciHub were updated, but this will solve the problem only by doing a minor version release.

1 Like

Hi, I am getting the error: “Failed to retrieve the product list from Scientific Data Hub”.
Is it still an open issue?

Hi I encounter the same issue on version 8.0.0. I test the exact same credential works well on version 7 (Win10 machine). The same error happens on both Win10 and Ubuntu 18.04 machines. I also notice credentials for USGS not work as well.

seems both URLs for USGS and SciHub are failed if directly open in browser:


Detailed error message below:

scihub

.seadas.seadas.reader] starting from SystemClassLoader[274 modules]; see http://wiki.netbeans.org/DevFaqModuleCCE
FINE [global]: Details: [expectContinueEnabled=false, proxy=null, localAddress=null, cookieSpec=null, redirectsEnabled=true, relativeRedirectsAllowed=true, maxRedirects=50, circularRedirectsAllowed=false, authenticationEnabled=true, targetPreferredAuthSchemes=null, proxyPreferredAuthSchemes=null, connectionRequestTimeout=300000, connectTimeout=300000, socketTimeout=300000, contentCompressionEnabled=true]
FINE [global]: HTTP GET https://scihub.copernicus.eu/apihub/search?rows=1&start=0&q=footprint%3A"Intersects(POLYGON((-81.6264%2041.1321%2C-83.1739%2041.1321%2C-83.1739%2042.8516%2C-81.6264%2042.8516%2C-81.6264%2041.1321)))"%20AND%20platformName%3ASentinel-3%20AND%20beginPosition%3A[2021-06-01T00%3A00%3A00.000Z%20TO%202021-06-03T23%3A59%3A59.000Z]%20AND%20endPosition%3A[2021-06-01T00%3A00%3A00.000Z%20TO%202021-06-03T23%3A59%3A59.000Z] returned 401
SEVERE [org.esa.snap.product.library.ui.v2.thread.AbstractRunnable]: Failed to retrieve the product list from ‘Scientific Data Hub’.
ro.cs.tao.datasource.QueryException: The supplied credentials are invalid!
at ro.cs.tao.datasource.remote.scihub.SciHubDataQuery.getCount(SciHubDataQuery.java:221)
at org.esa.snap.remote.products.repository.download.RemoteRepositoriesManager.downloadProductList(RemoteRepositoriesManager.java:368)
at org.esa.snap.remote.products.repository.download.RemoteProductsRepositoryProviderImpl.downloadProductList(RemoteProductsRepositoryProviderImpl.java:65)
at org.esa.snap.product.library.ui.v2.repository.remote.download.DownloadProductListTimerRunnable.execute(DownloadProductListTimerRunnable.java:80)
at org.esa.snap.product.library.ui.v2.repository.remote.download.DownloadProductListTimerRunnable.execute(DownloadProductListTimerRunnable.java:26)
[catch] at org.esa.snap.product.library.ui.v2.thread.AbstractRunnable.run(AbstractRunnable.java:39)
at java.lang.Thread.run(Thread.java:748)


and USGS:

FINE [global]: Details: [expectContinueEnabled=false, proxy=null, localAddress=null, cookieSpec=null, redirectsEnabled=true, relativeRedirectsAllowed=true, maxRedirects=50, circularRedirectsAllowed=false, authenticationEnabled=true, targetPreferredAuthSchemes=null, proxyPreferredAuthSchemes=null, connectionRequestTimeout=300000, connectTimeout=300000, socketTimeout=300000, contentCompressionEnabled=true]
FINE [global]: HTTP POST https://earthexplorer.usgs.gov/inventory/json/v/1.4.1/login returned 404
SEVERE [org.esa.snap.product.library.ui.v2.thread.AbstractRunnable]: Failed to retrieve the product list from ‘USGS’.
ro.cs.tao.datasource.QueryException: The request was not successful. Reason: Not Found
at ro.cs.tao.datasource.usgs.Landsat8Query.authenticate(Landsat8Query.java:160)
Caused: ro.cs.tao.datasource.QueryException
at ro.cs.tao.datasource.usgs.Landsat8Query.authenticate(Landsat8Query.java:165)
at ro.cs.tao.datasource.usgs.Landsat8Query.buildQueryUrl(Landsat8Query.java:254)
at ro.cs.tao.datasource.usgs.Landsat8Query.lambda$executeQuery$0(Landsat8Query.java:372)
Caused: ro.cs.tao.datasource.QueryException
at ro.cs.tao.datasource.usgs.Landsat8Query.lambda$executeQuery$0(Landsat8Query.java:395)
at java.lang.Iterable.forEach(Iterable.java:75)
at ro.cs.tao.datasource.usgs.Landsat8Query.executeQuery(Landsat8Query.java:368)
at ro.cs.tao.datasource.usgs.Landsat8Query.getCountImpl(Landsat8Query.java:92)
Caused: ro.cs.tao.datasource.QueryException
at ro.cs.tao.datasource.usgs.Landsat8Query.getCountImpl(Landsat8Query.java:94)
at ro.cs.tao.datasource.DataQuery.getCount(DataQuery.java:199)
at org.esa.snap.remote.products.repository.download.RemoteRepositoriesManager.downloadProductList(RemoteRepositoriesManager.java:368)
at org.esa.snap.remote.products.repository.download.RemoteProductsRepositoryProviderImpl.downloadProductList(RemoteProductsRepositoryProviderImpl.java:65)
at org.esa.snap.product.library.ui.v2.repository.remote.download.DownloadProductListTimerRunnable.execute(DownloadProductListTimerRunnable.java:80)
at org.esa.snap.product.library.ui.v2.repository.remote.download.DownloadProductListTimerRunnable.execute(DownloadProductListTimerRunnable.java:26)
[catch] at org.esa.snap.product.library.ui.v2.thread.AbstractRunnable.run(AbstractRunnable.java:39)
at java.lang.Thread.run(Thread.java:748)

Consider to update SNAP to 8.0.3.
The issue with SciHub is known and fixed, but not yet released.
(SNAP-1435)

thank you for the reply. yes I tried 8.0.3 as well after my previous post, same issue. Hope this will be fixed in 8.0.4?
Chui,