No valid orbit file found with "Sentinel Precise" but with "Restituted" even though they are no longer available

vvaroquaux · December 14, 2021, 11:47am

I have a S1 product ingested in May 2021.

I try to apply Orbit File with Type=Sentinel Precise (Auto Download), but no valid orbit is found.
Then, I try with Type=Sentinel Restituted (Auto Download), it fails because orbits are no longer available. It automatically retries with Type=Sentinel Precise (Auto Download) and it works !

First try => Fail
gpt -x -J-Xmx9000m Apply-Orbit-File -Ssource=test.dimap.dim -PorbitType='Sentinel Precise (Auto Download)'

WARNING: org.esa.s1tbx.orbits.gpf.ApplyOrbitFileOp: No valid orbit file found for 26-MAY-2021 08:53:13.000000
Orbit files may be downloaded from https://scihub.copernicus.eu/gnss/odata/v1/
and placed in /home/varoquaux/.snap/auxdata/Orbits/Sentinel-1/POEORB/S1B/2021/05
OpenSearch: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:B AND producttype:AUX_RESORB AND beginposition:[2021-05-01T00:00:000Z TO 2021-05-31T24:00:000Z]
OpenSearch: 0 total results on 1 pages.
OpenSearch: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:B AND producttype:AUX_RESORB AND beginposition:[2021-06-01T00:00:000Z TO 2021-06-31T24:00:000Z]
OpenSearch: 0 total results on 1 pages.
No valid orbit file found for 26-MAY-2021 08:53:13.000000
Orbit files may be downloaded from https://scihub.copernicus.eu/gnss/odata/v1/
and placed in /home/varoquaux/.snap/auxdata/Orbits/Sentinel-1/POEORB/S1B/2021/05
org.esa.snap.core.gpf.OperatorException: No valid orbit file found for 26-MAY-2021 08:53:13.000000
Orbit files may be downloaded from https://scihub.copernicus.eu/gnss/odata/v1/

Second try => fail => Fallback => Success !!
gpt -x -J-Xmx9000m Apply-Orbit-File -Ssource=test.dimap.dim -PorbitType='Sentinel Restituted (Auto Download)'

OpenSearch: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:B AND producttype:AUX_POEORB AND beginposition:[2021-05-01T00:00:000Z TO 2021-05-31T24:00:000Z]
OpenSearch: 31 total results on 1 pages.
OpenSearch: https://scihub.copernicus.eu/gnss/search?q=platformname:Sentinel-1 AND platformnumber:B AND producttype:AUX_POEORB AND beginposition:[2021-05-01T00:00:000Z TO 2021-05-31T24:00:000Z]
WARNING: org.esa.s1tbx.orbits.gpf.ApplyOrbitFileOp: Using Sentinel Precise /home/varoquaux/.snap/auxdata/Orbits/Sentinel-1/POEORB/S1B/2021/05/S1B_OPER_AUX_POEORB_OPOD_20210615T111837_V20210525T225942_20210527T005942.EOF.zip instead

It seems that in the first case, an error is thrown before the call to OpenSearch::download as there is no trace at all. But as the error is caught but not displayed, I don’t know why.

SentinelPODOrbitFile.java:

            try {
                final GnssOrbitFileDownloader gnssOrbitFileDownloader = new GnssOrbitFileDownloader();
                orbitFile = gnssOrbitFileDownloader.download(localFolder, "Sentinel-1", missionPrefix,
                        orbitType, year, month, day, stateVectorTime);
            } catch(Exception e) {
                // try next
            }
        }```

0yves0 · December 16, 2021, 3:20pm

Hi all !

I encountered the same behavior on SNAP 8.0 when invoking the Apply-Orbit-File operator with “Sentinel Precise (Auto Download)” as argument from Python code and S1 product ingested in March & April 2021.

I printed the stack trace of the caught exception @vvaroquaux mentioned in the latest version of the s1tbx source code. Here is the output :

java.lang.Exception: Unsupported orbit type Sentinel Precise (Auto Download)
        at org.esa.s1tbx.io.orbits.sentinel1.GnssOrbitFileDownloader.convertOrbitType(GnssOrbitFileDownloader.java:125)
        at org.esa.s1tbx.io.orbits.sentinel1.GnssOrbitFileDownloader.constructQuery(GnssOrbitFileDownloader.java:108)
        at org.esa.s1tbx.io.orbits.sentinel1.GnssOrbitFileDownloader.download(GnssOrbitFileDownloader.java:46)
        at org.esa.s1tbx.io.orbits.sentinel1.SentinelPODOrbitFile.retrieveOrbitFile(SentinelPODOrbitFile.java:88)
        at org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp.updateOrbits(ApplyOrbitFileOp.java:269)
        at org.esa.s1tbx.sar.gpf.orbits.ApplyOrbitFileOp.initialize(ApplyOrbitFileOp.java:203)
        at org.esa.snap.core.gpf.internal.OperatorContext.initializeOperator(OperatorContext.java:528)
        at org.esa.snap.core.gpf.internal.OperatorContext.getTargetProduct(OperatorContext.java:298)
        at org.esa.snap.core.gpf.Operator.getTargetProduct(Operator.java:385)
        at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:77)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:199)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initOutput(GraphContext.java:166)
        at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:85)
        at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:58)
        at org.esa.snap.core.gpf.graph.GraphProcessor.executeGraph(GraphProcessor.java:118)
        at org.esa.snap.core.gpf.main.DefaultCommandLineContext.executeGraph(DefaultCommandLineContext.java:86)
        at org.esa.snap.core.gpf.main.CommandLineTool.executeGraph(CommandLineTool.java:547)
        at org.esa.snap.core.gpf.main.CommandLineTool.runGraph(CommandLineTool.java:391)
        at org.esa.snap.core.gpf.main.CommandLineTool.runGraphOrOperator(CommandLineTool.java:287)
        at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:188)
        at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:121)
        at org.esa.snap.core.gpf.main.GPT.run(GPT.java:60)
        at org.esa.snap.core.gpf.main.GPT.main(GPT.java:37)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
        at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
        at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
        at org.esa.snap.runtime.Launcher.main(Launcher.java:31)

It seems there is a incompatibility between the String values the Apply-Orbit-File operator accepts (“Sentinel Precise (Auto Download)”, “Sentinel Restituted (Auto Download)”, etc) and the String values handled by the GnssOrbitFileDownloader class (“Sentinel Precise”, “Sentinel Restituted”) to download orbit files.

    // SentinelPODOrbitFile.java
    public final static String RESTITUTED = "Sentinel Restituted";
    public final static String PRECISE = "Sentinel Precise";

    // GnssOrbitFileDownloader.java
    private static final String POEORB = "AUX_POEORB";
    private static final String RESORB = "AUX_RESORB";

    private static String convertOrbitType(final String orbitType) throws Exception {
        switch (orbitType) {
            case RESORB:
            case SentinelPODOrbitFile.RESTITUTED:
                return RESORB;
            case POEORB:
            case SentinelPODOrbitFile.PRECISE:
                return POEORB;
            default:
                throw new Exception("Unsupported orbit type " + orbitType);
        }
    }

    // ApplyOrbitFileOp.java
    @Parameter(valueSet = {SentinelPODOrbitFile.PRECISE + " (Auto Download)", SentinelPODOrbitFile.RESTITUTED + " (Auto Download)",
            DorisOrbitFile.DORIS_POR + " (ENVISAT)", DorisOrbitFile.DORIS_VOR + " (ENVISAT)" + " (Auto Download)",
            DelftOrbitFile.DELFT_PRECISE + " (ENVISAT, ERS1&2)" + " (Auto Download)",
            PrareOrbitFile.PRARE_PRECISE + " (ERS1&2)" + " (Auto Download)",
            K5OrbitFile.PRECISE },
            defaultValue = SentinelPODOrbitFile.PRECISE + " (Auto Download)", label = "Orbit State Vectors")
    private String orbitType = null;

Is it a bug in the s1tbx or am I doing something wrong ? I can’t find anything about this subject on JIRA platform.

Thx

Yves

vvaroquaux · December 17, 2021, 1:44pm

Good catch !
Everywhere else (as far as I searched), orbitType value is compared using contains() or startsWith(), but in convertOrbitType, it is compared using the whole value.

I suggest something like:

    private static String convertOrbitType(final String orbitType) throws Exception {
        if (orbitType.startsWith(RESORB) || orbitType.startsWith(SentinelPODOrbitFile.RESTITUTED)) {
                return RESORB;
        }
        if (orbitType.startsWith(POEORB) || orbitType.startsWith(SentinelPODOrbitFile.PRECISE)) {
                return POEORB;
        }
        throw new Exception("Unsupported orbit type " + orbitType);
    }

A similar problem may occur in OrbitFileScraper.java where the orbitType value is used as is in the remotePath url (but I don’t know this API, so it may not be an error):


    public static class Step extends OrbitFileScraper {
        private static final String stepS1OrbitsUrl = "http://step.esa.int/auxdata/orbits/Sentinel-1/";

        public Step(final String orbitType) {
            super(orbitType);
            this.baseURL = stepS1OrbitsUrl;
        }

        @Override
        RemoteOrbitFile[] getFileURLs(final String mission, final int year, final int month) {
            final String monthStr = StringUtils.padNum(month, 2, '0');

            String remotePath = baseURL + orbitType + '/' + mission + '/' + year + '/' + monthStr + '/';

            final List<RemoteOrbitFile> remoteOrbitFiles = getFileURLs(remotePath);

            return remoteOrbitFiles.toArray(new RemoteOrbitFile[0]);
        }
    }
}