AgileEngine - Data Engineer - Python

2 weeks ago

Indore, India AgileEngine Full time

Data Engineer - Python

Experience : 5+ Years

Employment Type : Full-time

Job Overview :


We are seeking a highly skilled Data Engineer with 5+ years of experience, specializing in Python, to join our team. You'll be instrumental in designing, building, and maintaining robust and scalable data pipelines, ensuring data quality, and enabling data-driven decision-making. This role requires strong proficiency in Python for data manipulation, ETL processes, and integration within various data ecosystems.

Key Responsibilities :


- Design, develop, and optimize scalable data pipelines using Python to extract, transform, and load (ETL/ELT) data from diverse sources into data warehouses or data lakes.

- Write clean, efficient, and well-documented Python code for data processing, automation, and integration tasks.

- Collaborate with data scientists, analysts, and other engineering teams to understand data requirements and translate them into robust technical solutions.

- Implement best practices for data quality, data governance, and data integrity throughout the data lifecycle.

- Monitor, troubleshoot, and optimize existing data pipelines for performance, reliability, and cost-efficiency.

- Develop and maintain automated processes for data validation, reconciliation, and error handling.

- Work with various database technologies and big data platforms (e.g., SQL, NoSQL, data warehouses, cloud data services).

- Contribute to the continuous improvement of data engineering processes, tools, and methodologies.

Required Skills :


- 5+ years of professional experience as a Data Engineer.

- Expert-level proficiency in Python for data manipulation, scripting, and building data pipelines.

- Strong experience with ETL/ELT concepts and tools/frameworks in a Python ecosystem (e.g., Pandas, Dask, PySpark, Apache Airflow).

- Solid understanding and hands-on experience with relational databases (SQL) and data warehousing

principles.

- Experience with cloud platforms and their data services (e.g., AWS, Azure, GCP - including services like S3, Redshift, Glue, Data Factory, BigQuery, etc.).

- Familiarity with version control systems like Git.

- Strong analytical and problem-solving skills, with an eye for detail.

- Excellent communication and collaboration skills to work effectively within cross-functional teams.

Preferred Qualifications :


- Experience with Big Data technologies (Apache Spark, Hadoop, Kafka).

- Hands-on experience with NoSQL databases (MongoDB, Cassandra, DynamoDB).

- Familiarity with data modeling techniques (dimensional modeling, 3NF).

- Experience with CI/CD pipelines for data solutions.


(ref:hirist.tech)

  • AgileEngine - Windows VM Engineer - Cisco Administration

    2 weeks ago

    Indore, India AgileEngine Full time

    Role : Windows VM EngineerKey skills : VMware, VM Sphere, CISCO, Windows Administrator, AWS Cloud ServicesExperience : 5+ YearsNotice Period : Immediate- 30 DaysResponsibilities :- Manage and administer VMware vSphere environments, including VM provisioning, resource allocation, performance monitoring, and troubleshooting.- Serve as a Subject Matter Expert...

  • AgileEngine - BizTalk Developer

    2 weeks ago

    Indore, India AgileEngine Full time

    Role : BizTalk Developer - Immediate Joiners OnlyExperience : 5+ YearsNotice Period : Immediate OnlyResponsibilities :- Design, develop, and deploy BizTalk Server solutions, including orchestration, mapping, schemas, and pipelines.- Develop and maintain .NET applications and services to support BizTalk Server integrations.- Experience in EDI (Electronic Data...

  • Python Lead Engineer

    4 days ago

    Indore, Madhya Pradesh, India AgileEngine Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Lead Python Engineer - WFHTHIS IS FULLY REMOTE WORKING OPPORTUNITYWe someone who can join in less than 1 month.If you are interested and fulfill the below mentioned criteria then please share below information.EMAIL IDYears of relevant experienceCCTC/ECTCNotice periodUpdated resume mentioning all the mentioned skills. Also, experience in financial domain is...

  • AgileEngine - IICS/CAI Integration Engineer

    2 weeks ago

    Indore, India AgileEngine Full time

    Role : IICS CAI Integration Engineer Informatica.Experience : 5 to 12 Years.Mandatory Skills : Integration Patterns, Cloud Application Integration (CAI), Informatica.Location : PAN India.Notice Period : Immediate30 Days.Primary Responsibilities :- Design, develop, and implement complex integrations using Informatica IICS CAI.- Develop and maintain CAI...

  • Infrastructure & Networking Engineer (Middle) ID35138

    2 days ago

    Indore, Madhya Pradesh, India AgileEngine Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    AgileEngine is one of the Inc. 5000 fastest-growing companies in the US and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15 industries change the lives of millions. If you like a challenging environment where you're working with the best and are encouraged to learn and...

  • AI Engineer

    2 weeks ago

    Indore, India Agileengine Full time

    About the RoleWe're looking for an experienced Senior AI Engineer to join our elite consulting team. You'll work on diverse internal R&D initiatives and high-stakes client projects, delivering production-ready AI systems that solve real business problems. This role combines deep technical expertise with client-facing responsibilities. You'll architect...

  • Full Stack Engineer

    2 weeks ago

    Indore, India Agileengine Full time

    ABOUT THE ROLE We are looking for a Senior Full Stack Engineer (Ruby on Rails + React) to join our fast-paced, collaborative team on a project with an expected duration of 6 months. Youll work closely with product owners, designers, QA, and developers to build solutions that drive business impact, participating in all stages of product development from...

  • Data Engineer

    1 week ago

    Indore, India Orange Data Tech Full time

    Key Responsibilities: Design, develop, and maintain ETL pipelines and data workflows.- Build and optimize data models, warehouses, and lakes .- Work with structured and unstructured datasets for analytics and reporting.- Collaborate with cross-functional teams including Data Science and Product.- Ensure data quality, governance, and security across...

  • Data Engineer

    14 hours ago

    Indore, Madhya Pradesh, India Orange Data Tech Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Key Responsibilities:Design, develop, and maintain ETL pipelinesand data workflows.- Build and optimize data models, warehouses, and lakes.- Work with structured and unstructured datasets for analytics and reporting.- Collaborate with cross-functional teams including Data Science and Product.- Ensure data quality, governance, and securityacross systems.-...

  • InRiver PIM Operations Manager

    2 days ago

    Indore, Madhya Pradesh, India AgileEngine Full time ₹ 10,000 - ₹ 60,000 per year

    InRiver PIM Operations Manager :Oversee the daily operations of the InRiver PIM system, ensuring data accuracy, completeness, and consistency across all channels. Manage and execute product data updates, including data enrichment, cleansing, and validation within the InRiver platform. Develop and maintain data governance policies and procedures within the...