Senior Data Engineer

2 weeks ago

Mumbai Pune, India Go Digital Technology Consulting Full time ₹ 8,00,000 - ₹ 24,00,000 per year

Role: Senior Data Engineer

Location: Mumbai & Pune

Experience: 3yrs to 8yrs

Technologies / Skills: Advanced SQL, Python and associated libraries like Pandas, NumPy etc., Pyspark , Shell scripting, Data-Modelling, Big data, Hadoop, Hive, ETL pipelines.

Responsibilities:
• Proven success in communicating with users, other technical teams, and

senior management to collect requirements, describe data modeling decisions and develop data

engineering strategy.


• Ability to work with business owners to define key business requirements and convert to user stories with required technical specifications.


• Communicate results and business impacts of insight initiatives to key stakeholders to

collaboratively solve business problems.


• Working closely with the overall Enterprise Data & Analytics Architect and Engineering practice leads to ensure adherence with the best practices and design principles.


• Assures quality, security and compliance requirements are met for supported area.


• Design and create fault-tolerance data pipelines running on cluster


• Excellent communication skills with the ability to influence client business and IT teams


• Should have design data engineering solutions end to end. Ability to come up with scalable and

modular solutions.

Required Qualification:


• 3+ years of hands-on experience Designing and developing Data Pipelines for Data Ingestion or

Transformation using Python (PySpark)/Spark SQL in AWS cloud


• Experience in design and development of data pipelines and processing of data at scale.


• Advanced experience in writing and optimizing efficient SQL queries with Python and Hive handling Large Data Sets in Big-Data Environments


• Experience in debugging, tunning and optimizing PySpark data pipelines


• Should have implemented concepts and have good knowledge of Pyspark data frames, joins,

caching, memory management, partitioning, parallelism etc.


• Understanding of Spark UI, Event Timelines, DAG, Spark config parameters, in order to tune the

long running data pipelines.


• Experience working in Agile implementations


• Experience with building data pipelines in streaming and batch mode.


• Experience with Git and CI/CD pipelines to deploy cloud applications


• Good knowledge of designing Hive tables with partitioning for performance.

Desired Qualification:


• Experience in data modelling.


• Hands on creating workflows on any Scheduling Tool like Autosys, CA Workload Automation.


• Proficiency in using SDKsfor interacting with native AWS services.


• Strong understanding of concepts of ETL, ELT and data modeling.

  • Senior Data Engineer

    2 weeks ago

    Pune, India Data Axle Full time

    Job Description About Data Axle: Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set up a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology...

  • Senior Data Engineer

    5 days ago

    Mumbai, Pune, India Iquest Management Consultants Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job Title: Senior Data EngineerExperience: 8+ YearsLocation : Pune & MumbaiEmployment Type: Full-timeJob Description:We are looking for a Senior Data Engineer who is responsible for the design and development of scalable data pipelines and integrations to support continual increases in data volume and complexity. Work with analytics and business teams to...

  • Data Engineer

    4 weeks ago

    Pune, India Jash Data Sciences Full time

    Do you love solving real-world data problems with the latest and best techniques? And having fun while solving them in a team Then come join our high-energy team of passionate data people. Jash Data Sciences is the right place for you. We are a cutting-edge Data Sciences and Data Engineering startup based in Pune, India. We believe in continuous learning and...

  • Senior Data Engineer

    2 weeks ago

    Pune, India Dataceria Full time

    Company: Dataceria software solutions Pvt ltd.Job Position: Senior Data engineer _snowflakeLocation: PuneTimings to align with UK and USA timingsHybrid: 3 days in NT office PuneInterview stages: 1, initial screening, 2 stages client technical interviewStart date: asapDataceria is looking for a Senior Data Engineer to join our growing Data Delivery team.This...

  • Senior Data Engineer

    2 days ago

    Pune, Maharashtra, India Orcapod Consulting Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Senior Data EngineerSenior Data EngineersBuild and maintain data pipelines to support AI initiatives. Ensure data quality, integrity, and availability.Design and build ETL/ELT pipelinesSQL/NoSQL, Spark/Kafka, Airflow, ETL/ELT pipeline development; data governance, schema enforcementVector databases (Milvus, Pinecone), data lineage toolsDesign and build...

  • Senior Data Engineer

    2 weeks ago

    Pune, India Dataceria Full time

    Company: Dataceria software solutions Pvt ltd.Job Position: Senior Data engineer _snowflakeLocation: PuneTimings to align with UK and USA timingsHybrid: 3 days in NT office PuneInterview stages: 1, initial screening, 2 stages client technical interviewStart date: asapDataceria is looking for a Senior Data Engineer to join our growing Data Delivery team.This...

  • Senior Data Engineer

    2 weeks ago

    pune, India Dataceria Full time

    Company: Dataceria software solutions Pvt ltd.Job Position: Senior Data engineer _snowflakeLocation: PuneTimings to align with UK and USA timingsHybrid: 3 days in NT office PuneInterview stages: 1, initial screening, 2 stages client technical interviewStart date: asapDataceria is looking for a Senior Data Engineer to join our growing Data Delivery team.This...

  • Senior Data Engineer

    2 weeks ago

    Pune, India Dataceria Full time

    Company: Dataceria software solutions Pvt ltd.Job Position: Senior Data engineer _snowflakeLocation: PuneTimings to align with UK and USA timingsHybrid: 3 days in NT office PuneInterview stages: 1, initial screening, 2 stages client technical interviewStart date: asapDataceria is looking for a Senior Data Engineer to join our growing Data Delivery team.This...

  • Senior Data Engineer

    2 weeks ago

    Pune, India Dataceria Full time

    Company: Dataceria software solutions Pvt ltd.Job Position: Senior Data engineer _snowflakeLocation: PuneTimings to align with UK and USA timingsHybrid: 3 days in NT office PuneInterview stages: 1, initial screening, 2 stages client technical interviewStart date: asapDataceria is looking for a Senior Data Engineer to join our growing Data Delivery team.This...

  • Senior data engineer

    6 days ago

    Pune, India Dataceria Full time

    Company: Dataceria software solutions Pvt ltd.Job Position: Senior Data engineer _snowflakeLocation: PuneTimings to align with UK and USA timingsHybrid: 3 days in NT office PuneInterview stages: 1, initial screening, 2 stages client technical interviewStart date: asapDataceria is looking for a Senior Data Engineer to join our growing Data Delivery team.This...