I was considering something like this to have a local (or local networked) cache store for S3, I am doing 99,9% of readings, so it could be nice to put the cache in the S3-layer.
I think you can setup nginx proxy with cache. Unfortunately, that's easy only for public files and authenticated requests won't save you much (for that you can also write a small script to authenticate yourself if there's a cache hit).
S3Proxy cannot provide more that the S3 API allows, such as byte range updates, although some features can be enabled by bucket-level policies. S3Proxy can emulate features that other providers lack like public buckets and could provide other features like bucket-in-hostname. Apache jclouds provides the underlying object store compatibility so S3Proxy will benefit from improvements to it.
This is great timing for me, currently working on trying to iron out some bugs in a project with S3, and this helps me eliminate one line of inquiry. Nice project!
Surely the latency cost would be trivial compared to the inherent latency of using an off-site storage anyway?
I'm not sure why you're both asking for it, bit strange given how tiny the cost of using a generic interface is compared to the cost of sending a file over the internet.
It seems silly for every client to implement support for every single object store out there. Libraries like jcloud can handle it, but then it's tricky if your client is not written in Java.
Also, sometimes it's not trivial to add support to existing projects. For example, you can use s3fs with s3proxy to store your files on non-s3 object stores.
Does anyone knows if that exists?