Our client is a EU-based online large scaled, cloud first online marketplace, a part of the major group consoldating similar competence entities around the world.
They are developing a C2B2C marketplace in one of the growing European markets with sophisticated logic that allows users to buy and sell second hand items.
We are looking for a Data Engineer that will join the data engineering team and will help with data clean-up and migration tasks.
Due to a legal requirement the client has a task to migrate all their images to another storage (currently all the client's images are stored in an S3 bucket and they take up around 400TB) and clean up conversations that are linked to some groups of users.
The key projects of the Data Engineering team still need to be delivered as fast as possible (e.g., Streaming, New Data Lake, Deprecation of Old Data Lake). The volume of information to be removed is substantial, and it needs to be done in a cost-efficient way and has a minimal impact on production systems.
