AmplifAI - Senior ETL Consultant - Python/PySpark

2 months ago

Hyderabad, India AMPLIFAI PERFORMANCE MANAGEMENT PRIVATE LIMITED Full time

About the job :

Job Description :

Are you passionate about leveraging cutting-edge technologies in Data Engineering to drive innovation in a Series A product startup? AmplifAI is seeking an ETL Developer to join our dynamic team, contributing to the enhancement of our platform that enables enterprises to track and manage the performance of global customer service teams.

As a developer in the ETL Team, you will play a vital role in implementing, optimizing, and maintaining ETL processes critical to the company's backend operations.

Responsibilities :

Azure Orchestration and Hosting :

- Implement Azure Logic App flows to seamlessly ingest data from clients' systems into the cloud data store server.

- Configure Azure Functions using NodeJS and SQL to execute key functions, including extracting data from APIs and data objects, transforming it into predefined schemas, and efficiently loading each dataset into the cloud data store.

Python-Based ETL Pipeline :

- Implement data processing functions with NodeJS in our ETL pipeline, extracting data from a range of sources and formats and efficiently inserting it into SQL databases.

- Parameterize the ETL pipeline to improve data processing latency and significantly reduce man-hours.

Apache Spark Development and Maintenance Docker Deployment :

- Design and implement ETL processes using Apache Spark in standalone mode for enhanced data processing capabilities.

- Deploy and manage ETL workflows in Docker containers, ensuring scalability, flexibility, and ease of deployment.

Required Skills and Qualifications :

Education : Bachelor's degree in computer science, Information Technology, or a related field.

Experience :

- Proven experience in implementing ETL processes within cloud environments, particularly Azure.

- Hands-on experience with Azure Logic Apps, Azure Functions, and NodeJS.

- Strong proficiency in Python, with experience in developing Python-based ETL pipelines.

- Experience with Apache Spark and PySpark.

- Experience with Docker for deployment and Git CICD pipelines.

- Comfortable with Windows and Linux Operating systems.

Collaboration and Communication :

- Excellent communication and interpersonal skills.

- Ability to collaborate with cross-functional teams and adapt to a fast-paced startup environment.

If you are excited about the intersection of Big Data and Analytics, and are ready to contribute to a startup environment, we encourage you to apply for this challenging and impactful position.

(ref:hirist.tech)

  • Senior Full Stack Developer

    4 weeks ago

    Hyderabad, Telangana, India AmplifAI Full time

    We are seeking an experienced Senior Full Stack Developer to join our team at AmplifAI. **Job Summary:**The ideal candidate will have a solid understanding of the full software design life cycle and experience in designing, coding, testing, and consistently managing applications. They should be comfortable coding in multiple languages and have the ability to...

  • Senior Software Architect for Cloud Applications

    4 weeks ago

    Hyderabad, Telangana, India AmplifAI Full time

    About UsAt AmplifAI, we are revolutionizing the field of artificial intelligence by leveraging cutting-edge technology and innovative thinking.

  • Senior Dotnet Software Engineering Professional

    4 weeks ago

    Hyderabad, Telangana, India AmplifAI Full time

    AmplifAI is looking for a skilled Senior Dotnet Developer to join our team. We offer an estimated salary of $120,000 - $180,000 per year.About the RoleThe ideal candidate will have experience in designing, coding, testing, and managing applications using .NET Core or relevant technologies. They should be comfortable working with multiple programming...

  • Etl pyspark developer

    2 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Job Title ETL PYSpark Developer Relevant Experience (in Yrs) 6-8 years Location- Hyd/Chn/Kol/Ahm/Gurgoan Technical/Functional Skills Must have Experience: 1. At least 8+ years of experience architecting and implementing complex ETL pipelines preferably with Spark toolset. 2. At least 4+ years of experience with Java particularly within the data space...

  • Etl pyspark developer

    2 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Job Title ETL PYSpark DeveloperRelevant Experience (in Yrs) 6-8 yearsLocation- Hyd/Chn/Kol/Ahm/GurgoanTechnical/Functional Skills Must have Experience:1. At least 8+ years of experience architecting and implementing complex ETL pipelines preferably with Spark toolset.2. At least 4+ years of experience with Java particularly within the data space3. Technical...

  • ETL PYSpark Developer

    2 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Job Title ETL PYSpark DeveloperRelevant Experience (in Yrs) 6-8 yearsLocation- Hyd/Chn/Kol/Ahm/GurgoanTechnical/Functional Skills Must have Experience:1. At least 8+ years of experience architecting and implementing complex ETL pipelines preferably with Spark toolset.2. At least 4+ years of experience with Java particularly within the data space3. Technical...

  • ETL PYSpark Developer

    2 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Job Title ETL PYSpark Developer Relevant Experience (in Yrs) 6-8 years Location- Hyd/Chn/Kol/Ahm/Gurgoan Technical/Functional Skills Must have Experience: 1. At least 8+ years of experience architecting and implementing complex ETL pipelines preferably with Spark toolset. 2. At least 4+ years of experience with Java particularly within the data...

  • ETL PYSpark Developer

    2 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Job Title ETL PYSpark DeveloperRelevant Experience (in Yrs) 6-8 yearsLocation- Hyd/Chn/Kol/Ahm/GurgoanTechnical/Functional Skills Must have Experience: 1. At least 8+ years of experience architecting and implementing complex ETL pipelines preferably with Spark toolset. 2. At least 4+ years of experience with Java particularly within the data space3....

  • ETL PYSpark Developer

    2 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Job Title ETL PYSpark DeveloperRelevant Experience (in Yrs) 6-8 yearsLocation- Hyd/Chn/Kol/Ahm/GurgoanTechnical/Functional Skills Must have Experience: 1. At least 8+ years of experience architecting and implementing complex ETL pipelines preferably with Spark toolset. 2. At least 4+ years of experience with Java particularly within the data space3....

  • ETL PYSpark Developer

    2 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Job Title ETL PYSpark Developer Relevant Experience (in Yrs) 6-8 years Location- Hyd/Chn/Kol/Ahm/Gurgoan Technical/Functional Skills Must have Experience: 1. At least 8+ years of experience architecting and implementing complex ETL pipelines preferably with Spark toolset. 2. At least 4+ years of experience with Java particularly within the data space 3....

  • Senior .NET Software Development Team Lead

    4 weeks ago

    Hyderabad, Telangana, India AmplifAI Full time

    Job OverviewAmplifAI is seeking an experienced Senior .NET Software Development Team Lead to join our team. This is a full-time position that offers a competitive salary of $150,000 - $180,000 per year.ResponsibilitiesLead the development team in designing and developing applications based on provided architecture.Collaborate with business analysts to gather...

  • Tata Consultancy Services | ETL PYSpark Developer

    2 weeks ago

    hyderabad, India Tata Consultancy Services Full time

    Job Title ETL PYSpark DeveloperRelevant Experience (in Yrs) 6-8 yearsLocation- Hyd/Chn/Kol/Ahm/GurgoanTechnical/Functional Skills Must have Experience: 1. At least 8+ years of experience architecting and implementing complex ETL pipelines preferably with Spark toolset. 2. At least 4+ years of experience with Java particularly within the data space3....

  • Data Engineer

    1 month ago

    Bangalore/Hyderabad, India ANRI Solutions HR Services Full time

    Job Description : We are seeking a skilled Data Engineer with 4-8 years of experience to join our dynamic team. The ideal candidate will have a strong background in Python and PySpark, and will be responsible for designing, developing, and maintaining scalable data pipelines and architectures.Responsibilities : - Design and implement data pipelines using...

  • Senior Data Engineer

    5 months ago

    Hyderabad, India Aeries Technology Group Full time

    Job Description :- Build data pipeline between source systems (RDS/SQL Server) to Snowflake on AWS- ETL data into AWS S3- ELT Data into Snowflake- Realtime data integration through CDC and AWS managed Kafka streams.- Embedded Tableau reporting- Data transformations using SQL, Python, PySpark, and DBT- Through understanding of data engineering principles...

  • Senior Data Engineer

    1 month ago

    Hyderabad, India Aeries Technology Group Full time

    Job Description :- Build data pipeline between source systems (RDS/SQL Server) to Snowflake on AWS- ETL data into AWS S3- ELT Data into Snowflake- Realtime data integration through CDC and AWS managed Kafka streams.- Embedded Tableau reporting- Data transformations using SQL, Python, PySpark, and DBT- Through understanding of data engineering principles...

  • Amplifai | senior dotnet developer

    3 weeks ago

    Hyderabad, India AmplifAI Full time

    The ideal candidate will be familiar with the full software design life cycle. They should have experience in designing, coding, testing and consistently managing applications They should be comfortable coding in a number of languages and have an ability to test code in order to maintain high-quality code.ResponsibilitiesDesign, code, test and manage various...

  • AmplifAI | Senior Software Engineer

    6 hours ago

    hyderabad, India AmplifAI Full time

    Full Stack .Net Developer - India Work harmoniously in a cross-functional team comprised of managers, supervisors, near-shore development team, systems personnel, network staff, and other developers to ensure the delivery of web solutions as planned. Develop and maintain application documentation and coding standards that are clear, accurate and detailed....

  • Data Engineer

    4 weeks ago

    Hyderabad, India People Impact Full time

    Role Summary : We are looking for an enthusiastic leader to manage a skilled data engineering team that focuses on developing and maintaining reusable data pipelines using Databricks (PySpark) and Google Cloud Platform (GCP). This role demands comprehensive project management and effective coordination with various stakeholders, including teams in the US and...

  • AmplifAI | Senior Software Engineer | hyderabad

    3 weeks ago

    hyderabad, India AmplifAI Full time

    Full Stack .Net Developer - IndiaWork harmoniously in a cross-functional team comprised of managers, supervisors, near-shore development team, systems personnel, network staff, and other developers to ensure the delivery of web solutions as planned.Develop and maintain application documentation and coding standards that are clear, accurate and detailed....

  • PySpark with Azure Databricks

    2 months ago

    hyderabad, India HTC Global Services Full time

    Dear Candidate,We have an opportunity of PySpark with DataBricks position with HTC Global ServicesPlease share your updated resume on with below details -We are Looking for Immediate or 15days joiners onlyCandidate's Full Name (As Per Aadhar) -Total Exp. -Rel. Exp. (in Pyspark) -Rel. Exp. (in Databricks) -Notice Period-If serving Notice or not working,...