A multi-stage Docker build where you separate pulling in dependencies from building the thing you want is as close as you're going to get.
Something like the following works well in practice:
1) pinned base image (e.g. Ubuntu LTS)
2) your own custom base image in a registry rebuilt whenever you want (e.g. with tools you need for debugging or available across all of your images)
3) your own runtime-specific base image, like a JDK one, can be used later both as a basis for development images with additional tooling, as well as for runtime images of your app
4) your own runtime-specific development images, like one that's based on the JDK image above + Maven, alongside any other development tooling you need
5) your multi-stage application image, where the first stage uses the development image to COPY in the dependency description files you need and then pull the dependencies, then does the build (layer cache takes care of reusing things where possible), and then the second stage is based on the runtime image (e.g. JDK) where you just copy your finished artifact (e.g. .jar file)
If you don't need or want to build your own images, you can fold steps 1-4 into just using upstream images off of Docker Hub or whatever you prefer, but in practice it works pretty okay across numerous stacks. Of course, it's also possible to easily have very high standards in regards to what you mean as "optimal", so Docker probably won't live up to that.
- I changed one source file, I need to rebuild the whole app
- I upped one dependency by a minor version
...
How long is the rebuild time? As I said, compare it to simple, standard make-based setup.
The problem is that layers are really — in the end — sequential, and even a simple change in an earlier one invalidates the latter cached layers.
I do not consider up-to-few-seconds for cases like above "very high standards", but it is a lot of work and care with Docker — especially as you need it not to mess up development or production environments as people skip versions and similar (if you use tricks like reading some config from a file).
Something like the following works well in practice:
If you don't need or want to build your own images, you can fold steps 1-4 into just using upstream images off of Docker Hub or whatever you prefer, but in practice it works pretty okay across numerous stacks. Of course, it's also possible to easily have very high standards in regards to what you mean as "optimal", so Docker probably won't live up to that.