Job Description: Tech Lead - Data Engineering
Company Overview: At Inallmedia, we are committed to developing innovative and high-impact solutions for our clients, leveraging advanced technologies to address complex business needs. We specialize in creating customized software solutions, and our data infrastructure plays a pivotal role in delivering actionable insights. As part of our ongoing growth, we are seeking a skilled Tech Lead to join our Data Engineering team and help drive the optimization and evolution of one of our most critical datasets: Usersession.
Position Overview: As a Tech Lead - Data Engineering, you will play a central role in enhancing the stability and efficiency of the Usersession dataset, which powers critical reports and analytics applications. You will also lead the integration of new datasets to support the growing needs of our organization. This is a high-impact role that requires both deep technical expertise and the ability to lead a team in building scalable, cost-efficient data systems.
You will be responsible for ensuring the reliability of the Usersession builder, optimizing the system for lower operational costs, and integrating new datasets to meet evolving business requirements. The role demands a leader who can effectively balance technical excellence with team collaboration to deliver top-tier solutions.
Key Responsibilities:
- Stability Enhancement & Cost Optimization:
- Lead efforts to enhance the stability of the Usersession builder and ensure its smooth operation.
- Focus on optimizing the system to reduce operational costs without sacrificing performance or reliability.
- Troubleshoot and resolve any issues within the data pipeline, ensuring minimal disruption to business operations.
- New Dataset Integration:
- Collaborate with internal stakeholders to identify new data requirements for the Usersession dataset.
- Architect and implement the seamless integration of new datasets, ensuring they align with the evolving needs of the business.
- Design solutions that are both scalable and maintainable, while optimizing for performance and cost.
- System Architecture & Design:
- Lead the design and development of scalable data systems and services to handle large volumes of data.
- Implement best practices in software engineering to ensure high quality, reliable, and maintainable systems.
- Guarantee data quality, consistency, and security across all integrated systems.
- Team Leadership & Collaboration:
- Provide technical leadership and mentorship to data engineers, fostering a culture of continuous learning and improvement.
- Work closely with cross-functional teams, including product managers, analysts, and other engineering teams, to align on technical solutions and business objectives.
- Conduct code reviews, promote adherence to coding standards, and guide the team to deliver high-quality code.
- Expertise in AWS, Spark, and Java:
- Lead the implementation of AWS services, particularly AWS EMR, to build scalable and cost-effective data pipelines.
- Leverage Apache Spark for large-scale data processing and analytics, ensuring high performance and scalability.
- Utilize Java expertise to develop robust and efficient software components that support the data infrastructure.
Required Qualifications:
- Work experience 10 + years
- Experience as a Tech Lead (3+ years)
- Extensive experience with Java, Apache Spark, and AWS EMR in large-scale data environments.
- Strong expertise in distributed systems, cloud infrastructure, and big data technologies.
- Proven experience optimizing and scaling data pipelines for high-volume datasets.
- Excellent leadership skills, with the ability to lead a team, mentor engineers, and drive technical projects.
- Strong problem-solving skills with a focus on performance optimization and cost reduction.
- Exceptional communication skills to convey complex technical concepts to non-technical stakeholders.
Preferred Qualifications:
- Experience with data modeling, ETL processes, and cloud-based data integration.
- Familiarity with other AWS services such as S3, Redshift, Lambda, and Kinesis.
- Previous experience leading a technical team in an agile, fast-paced environment.
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
Why Inallmedia? At Inallmedia, we foster an environment of innovation, collaboration, and growth. As a Tech Lead, you will have the opportunity to work on high-impact projects, lead a talented team, and contribute to the continued success of our data systems. You’ll work with cutting-edge technologies, solve challenging problems, and play a key role in shaping the future of our data infrastructure.