PySpark & ETL Data Engineer
7 days ago
Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. You welcome diversity at work and within the greater community. You aren't afraid to take risks. You thrive upon continuing education programs that your company sponsors to strengthen your skills and for you to become a thought leader ahead of the industry curve. You are excited about creating change because your skills can help the greater good of every customer, industry and community. Experience - 4-8 years CirrusLabs is seeking a skilled and experienced PySpark Data Engineer (ETL Lead) to join our growing data engineering team. As an ETL Lead, you will play a pivotal role in designing, developing, and maintaining robust data integration pipelines using PySpark and related technologies. You’ll work closely with data architects, analysts, and stakeholders to transform raw data into high-quality, actionable insights, enabling data-driven decision-making across the organization. This is an exciting opportunity for someone who is not only technically strong in PySpark and Python but also capable of leading data integration efforts for complex projects. Lead Data Integration Projects: Manage the data integration and ETL activities for enterprise-level data projects. Design and develop scalable and efficient PySpark scripts , both generic frameworks and custom solutions tailored to specific project requirements. Implement end-to-end ETL processes to ingest, clean, transform, and load data. Schedule and Automate ETL Processes: Integrate ETL workflows into automation tools and CI/CD pipelines. Optimize Data Processing: Optimize PySpark jobs for performance and resource efficiency. Monitor, troubleshoot, and resolve issues related to data processing and pipeline execution. Data Transformation and Curation: Transform raw data into consumable, curated data models suitable for reporting and analytics. Ensure data quality, consistency, and reliability throughout all stages of the ETL process. Collaborate with data architects, analysts, and business stakeholders to define requirements and deliver solutions. Contribute to the evolution of data engineering practices, frameworks, and standards. Provide guidance and mentorship to junior engineers on PySpark and ETL best practices. Develop and maintain technical documentation related to ETL processes, data flows, and solutions. Experience: ~5–8 years of professional experience in data engineering, ETL development, or related fields. ~ Proven experience leading data integration projects from design to deployment. ~ Strong hands-on experience with PySpark for building large-scale data pipelines. ~ Proficiency in Python , including writing efficient, reusable, and modular code. ~ Solid knowledge of SQL for data extraction, transformation, and analysis. ~ Data Engineering Expertise: ~ Experience working on data integration projects , such as data warehousing, data lakes, or analytics solutions. ~ Familiarity with processing structured and semi-structured data formats (e.g., Ability to transform and harmonize data from raw to curated layers. Familiarity with data pipeline orchestration tools (e.g., Experience with cloud platforms (e.g., AWS, Azure, GCP) is desirable.
-
PySpark & ETL Data Engineer
3 weeks ago
Hyderabad, India CirrusLabs Full timeWe are CirrusLabs. Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...
-
PySpark & ETL Data Engineer
3 weeks ago
Hyderabad, India CirrusLabs Full timeWe are CirrusLabs. Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...
-
PySpark & ETL Data Engineer
3 weeks ago
Hyderabad, India CirrusLabs Full timeWe are CirrusLabs . Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...
-
PySpark & ETL Data Engineer
3 weeks ago
Hyderabad, India CirrusLabs Full timeWe are CirrusLabs. Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...
-
PySpark & ETL Data Engineer
3 weeks ago
Hyderabad, India CirrusLabs Full timeWe are CirrusLabs . Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...
-
PySpark & ETL Data Engineer
3 weeks ago
Hyderabad, India CirrusLabs Full timeWe are CirrusLabs . Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...
-
Pyspark & Etl Data Engineer
3 weeks ago
Hyderabad, India Whatjobs IN C2 Full timeWe are CirrusLabs . Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...
-
PySpark & ETL Data Engineer
3 weeks ago
Hyderabad, India CirrusLabs Full timeWe are CirrusLabs. Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...
-
PySpark & ETL Data Engineer
3 weeks ago
Hyderabad, India CirrusLabs Full timeWe are CirrusLabs. Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...
-
PySpark & ETL Data Engineer
2 weeks ago
Hyderabad, India CirrusLabs Full timeWe are CirrusLabs. Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...
