About the role
ClearScale is looking for a Spark Cloud Data Engineer to participate in building cost-efficient scalable data lakes for a wide variety of customers, from small startups to large enterprises.
Usually, our projects fit into one of the categories (but not limited to them):
- Collect data from the IoT edge locations, store it in the Data Lake, orchestrate processes to ETL that data, slice it into the various data marts. Then put those data marts into the machine learning or BI pipelines
- Optimize pre-existing Spark clusters, find the bottlenecks in the code and/or architecture, plan and execute improvements
- Build a data delivery pipeline to ingest a high volume of the real-time streams, detect anomalies, slice into the window analytics, put those results in the NoSQL for the further dashboard consumption
What will you do on a daily basis:
- Analyze, scope, and estimate tasks, identify technology stack and tools
- Design and implement optimal architecture and migration plan
- Develop new and re-architecture solution modules, re-design and re-factor program code
- Examine performance and advise necessary architecture changes
- Communicate with the client on project-related issues
- Collaborate with in-house and external development and analytical team
ClearScale expects a successful candidate to have most of the following qualifications and skills::
- 1+ years of AWS hands-on experience
- 3-6 years of experience in Data, Cloud, or Software Engineer role, who has attained a degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field
- 1+ years of hands-on of Amazon EMR or AWS Glue
- 1+ years of hands-on Spark through SDK: Java, Scala, or Python
Additionally, it would make us smile even more if you have:
- 1+ years of hands-on Amazon Athena experience
- 1+ years of hands-on SQL experience
- AWS certifications, especially AWS Machine Learning Specialty
- Hands-on experience designing efficient architectures for high-load enterprise-scale applications or ‘big data’ pipelines
- Hands-on experience with general AWS infrastructure components
- Hands-on experience with message queuing, stream processing, and highly scalable ‘big data’ stores
- Knowledge and experience working with SQL and NoSQL databases
- Proven experience in re-design and re-architecting the large complex business applications
- Strong self-management and self-organizational skills
ClearScale can be a great fit for you if
- You'd like to work remotely with a flexible schedule
- You thrive in a small, dynamic, and agile team that encourages you to learn and grow
- You enjoy finding solutions to interesting problems and figuring out how things work
- You welcome having autonomy with complex tasks
- You are passionate about using your experience and expertise to inspire the team
We offer
# 1 Fair wage
- High hourly rate in USD
- Full-time, 40 hours per week contract. It’s your main and permanent place of work, not a freelance job
- Payments every 2 weeks
- Annual Rate Review
- Referral program
# 2 Independence
- From the office location or office conditions. The only requirement - fast and reliable internet.
- From big team buzz - we are the community of professional engineers, valuing each others' time and effort
- From corruption and bureaucracy - we operate in an honest and competitive environment and we are one of AWS's top 10 key partners.
# 3 Professional Development
- Work with innovative Silicon Valley companies and traditional American companies at the cutting edge of digital transformation
- We work with the newest technologies in AWS cloud and open-source tools like Jira, Confluence, Lucidchart, Slack, etc
- Paid AWS certifications: we provide training material, paid time off, and the examination itself
- Horizontal and vertical career growth - We keep growing and people keep growing with us
- A nice bonus if you are ready to start at ClearScale quickly
About ClearScale
ClearScale (headquartered in San Francisco, California, USA) - AWS Premier Consulting Partner has been offering a full range of professional cloud computing services for over 10 years, including architecture design, DevOps automation, refactoring and cloud-native applications development, integration, migration, solving all sorts of security issues (from just a security check to preventing cyber-attacks) and 24/7 technical support using the best-advanced technology.
We were the third company to gain a new AWS competence: Applied AI and Machine Learning Operations (MLOps). Less than 15 partners have this competency!
Since the very foundation of the company, we work 100% remotely from various cities and countries. We work on a high trust basis. The professional reputation of our employees is of the highest value.
