Mininmal SNAP for commandline only (cloud?)

Dear Developers and community,

I would like to install SNAP in a cloud environment - most preferably as Docker container, but for the least as slim command line version without any GUI stuff.

I know there’s already some information scattered in this forum about that, but I failed to find a comprehensive answer that would make sense to a non-developer like me.

I also know that there are no official Docker images or plans to provide some. But there are the efforts of @edwardpmorris who created a bunch of Dockerfiles for SNAP 2 and 3.

Anyways, I’m aiming for the slimmest, most recent version possible while maintaining the core functionality (S1/S2/S3 processing).

So my questions:

What’s the best way to go about that (yeah that’s the big one)

and

if/when I compile the code from source, do I need SNAP-desktop for the commandline functionality?

Again, apologies for my ignorance … but since I’m not a developer myself, my knowledge about Java and the SNAP construct is quite limited.

All tips are highly appreciated.

Best,

Val

Hi Val, the Docker containers I made are probably need updating, but I am happy to help out on the latest versions. At the time I kept most of the useful info in the GitHub README https://github.com/edwardpmorris/docker-snap

What’s the best way to go about that (yeah that’s the big one)
Without getting too much into Docker container options; you may want to think about how secure your container should be? How small/minimal it can it be? Will you use Python? A strategy for builds with different toolboxes, updates and components? From your post sounds like a slim, general, own-use container. So at first guess I would suggest latest Debian base image (you may try smaller base images), SNAP + S1, S2, S3 which will be > 2GB.

do I need SNAP-desktop for the commandline functionality?
I am not sure if it is feasible to compile without the graphical interface components (never tried), I too just basically used gpt. Maybe this is a question for the developers.

Bit busy the coming few days, but happy to answer questions and put some work into updated containers, cheers Ed.

1 Like

There is no need for snap-desktop if you just wan’t gpt. Also, you can drop all modules named “*-ui”.

Regards
Norman

1 Like

Thanks for the replies @edwardpmorris @forman.

I admit, I’m also fairly new to Docker … so I might not have the full scope of possibilities yet.
I did already go through your dockerfiles thoroughly before posting here. A couple of questions regarding that:
Are you creating separate dockerfiles for the different toolboxes?
Why use debian and not an ubuntu base image?

My idea was to generally keep it simple. Have one container which has all the SNAP functionality I need for processing S1 and S2 data (possible S3 as well to cover all the bases). Additionally I was reading in another post in this forum about the unattended toolbox update, which I would like to incorporate. I think that was something you also wanted to add to your dockerfiles.
So yes, slim, general and own-use is what I want.
I’m not sure what you mean with container security, maybe you can elaborate.

While filesize is generally not my first concern, I would like to drop everything associated with GUI as it will be running exclusively on the cloud.
As for python support, currently I’m not using it … but if it’s not too much hassle and space I’d include it to keep all options open.

Thanks!

Best regards,

Val

Yes, the logic was ‘run a single process’ per container (potentially on different cloud instances), it seemed unlikely that we would be running, for example both the S1 and S2 toolboxes at the same time, hence attempted to make single toolbox containers. Also pulling (downloading) containers from a repository is potentially slow, hence the emphasis on making them as small as possible.

At the time this was the ‘smallest’ fully functional Linux base image. Also, Ubuntu is based on Debian, so commands are essentially the same. There maybe smaller base images (such as Alpine Linux), but these need more involved setup to run SNAP (something on the list for the future).

I guess this is suitable if you will have one ‘permanent’ instance, where you do not pull (download) the Docker container each time you start an analysis (see above). Anyway should be no problem to create this container.

Again this is probably useful for a single instance. Need to look into it. Part of the container logic is that we can keep a very good track of exactly which versions are being used for bulk product production ensuring full transparency.

Best have a look at https://docs.docker.com/engine/security/security/. Also various articles available with simple advice. Depending on where you will run the container, you may consider taking advice from your IT support on this, as I am not an expert.

Yes I am keen to try this, we should coordinate with @forman to make a recipe that only includes gpt, snappy and the toolboxes.

No problem, previously this added only about 100Mb.

2 Likes

Thanks Norman. It would be great to get the ‘recipe’ set up to pull and compile only the components needed for a headless instance (gpt and/or snappy). Could this ‘headless version’ potentially be a candidate for official release in https://github.com/senbox-org ? cheers Ed.

Hi

I’ve added your requirement to our issue tracker (SNAP-705).
You should keep in mind that you will loos the functionality to update your SNAP installation when removing all desktop modules.
I’m not sure which modules are all needed, because I haven’t tried it, but removing all will also remove this functionality.

3 Likes

I could image that you might want to overlay a S1 scene over a S2, S3 or Landsat image. So you might need the other boxes too.
However, it depends on your use case.

1 Like

Many thanks for the in-depth explanations.

Essentially I’d like to have a single container for SNAP functionality which I can call within a processing chain. So I’m looking forward to the headless verision.

Thanks again all @edwardpmorris @forman @marpet

Hi Marco,

Just inquiring if the commandline installer is still planned / coming at some point.

Many thanks,

Val

It’s still on the agenda but not with a high priority.
I’ve raised the priority a bit and talked with the in-house guys which use it on our cluster. They support the idea too and would benefit from it.

3 Likes

Hi Marco,

Is there any plans for this feature to be released soon?
Thanks in advance.

1 Like