Data Engineer

Job Description

You’re an engineer with experience in cloud computing. You take pride in developing solutions that are not just functional – they are efficient, maintainable, and elegant. You probably have a wealth of knowledge to draw upon about best practices, as well as a suite of tools you use to analyze and monitor the quality of your code. But when it comes to understanding the efficiency of the cloud resources your solutions will use, where do you turn for guidance?

This is our mission at Yotascale. We build tools by engineers, for engineers.

About the Role

We are looking for an experienced data engineer to contribute to the growth of our code base and infrastructure. You will be working with a veteran team of engineers from both Lahore and Silicon Valley to define, scope and develop JVM-based, big data products. This will involve partnering with Product Managers to define application features; participating in end-to-end architectural reviews and broader architectural discussions with the engineering team; owning a meaningful part of the application code that directly impacts the success of the business; and resolving issues related to performance, scalability, data integrity & security. We believe in empowering every engineer to have a personal stake in the shaping of our applications; we don’t have a team of designers, and a separate team of doers – everyone contributes to both of these in line with their current level of experience. While we are a cross-functional team that supports one another, the main duties of this role are:

• Develop, improve, and deploy highly scalable distributed data pipelines that ingest, analyze, and store billions of time-series metrics and events to our downstream storage and analytics services
• Ensure the system delivers high throughput and low latency to enable the delivery of highly available, resilient, and durable services
• Build and maintain our data processing streams and frameworks

We use Spark for our data processing needs, and the majority of our application code is written in JVM-based languages such as Java or Kotlin. We use SQL databases both for our transactional database (Postgres) and our data warehouse (Clickhouse).  We make extensive use of AWS technologies to power our business.

What We Offer

• Flexibility to work remotely (or at our Lahore offices)
• Unlimited paid time off
• Every Yotascale employee is an owner of the business through equity grants
• Comprehensive mentorship and professional development programs
• Amazing medical coverage
• H1B Visa applications for senior high performing employees to travel, as needed, to work with the team in the US

Who You Are

As a values-driven company, even more than specific technical expertise we are looking for people who share our vision of creating a company culture that is focused on not just customer satisfaction but also on employee satisfaction and wellness. You have a strong sense of personal accountability to the work that you produce and know how to effectively communicate across teams. You are able to take feedback from your peers and your manager, to learn from your mistakes, and are hungry to continuously improve your software development, architecture, and leadership skills. While we will consider a range of candidates based on specific past experience, some of the skills and experience we are interested in are:

• 3+ years of professional software engineering experience with consumer-facing products
• Experience building, maintaining, and monitoring data pipelines
• Experience developing against a large-scale OLAP database with billions of records
• Strong knowledge of Python and PySpark (or equivalent in another language)
• Some familiarity with Java, Scala, or Kotlin and associated tools
• Demonstrated ability to test, monitor, and integrate your code with other production systems
• Solid CS fundamentals (data structures, algorithms, design patterns)
• Strong understanding of SQL and experience with at least one DBMS
• Bachelor’s degree or higher, in Computer Science or related field (or equivalent experience)
• Excellent communication and problem-solving skills

Who We Are

Yotascale builds tools that empower engineers, operations, and finance teams to understand their cloud utilization at a deep level. From allocating Kubernetes and EC2 instance costs across your organization, to using powerful machine learning routines to forecast usage patterns and make recommendations about your network architecture, Yotascale is becoming an increasingly critical part of companies’ infrastructure to track and understand the impact of their cloud footprint. Our leadership team comes from Amazon, Paypal and Ebay, with investors who also launched success stories like Twitch, Credit Karma, and Shopify.  We are a crucial component of some of the biggest names in the business such as Hulu and Zoom.