EverestDX - Data Engineer - ETL/Spark

6 days ago

Hyderabad, India EverestDX Full time

Job Purpose :As a Data Engineer - ETL/Spark, you will be responsible for designing, developing, and maintaining data integration workflows and pipelines.You will work with cross-functional teams to enable seamless data movement, transformation, and storage, ensuring data is accurate, reliable, and optimized for analytics and business intelligence use Responsibilities :ETL/ELT Development & Maintenance :- Design, develop, and maintain efficient ETL workflows using modern tools and frameworks.- Optimize and automate data pipelines to ensure high performance and scalability.- Monitor and troubleshoot ETL processes to ensure reliability and data quality.Big Data Engineering with Spark :- Develop and optimize Spark-based applications for large-scale data processing.- Implement distributed data processing workflows leveraging Hadoop, Spark, or other big data ecosystems.- Ensure Spark jobs are performance-tuned and cost-optimized.Data Management & Integration :- Work with structured and unstructured data from multiple sources, ensuring proper cleansing, transformation, and loading.- Integrate data from APIs, databases, cloud services, and third-party platforms.- Ensure compliance with data governance, security, and privacy standards.Collaboration & Business Enablement :- Partner with data analysts, data scientists, and business stakeholders to understand requirements and deliver data solutions.- Support data-driven initiatives by providing reliable and timely datasets.- Document processes, data flows, and technical specifications for reference and knowledge :- Bachelors/Masters degree in Computer Science, Information Technology, or related field.- 4- 8 years of hands-on experience in ETL development and data engineering.- Strong expertise in Apache Spark (PySpark/Scala/Java) for distributed data processing.- Proficiency with ETL tools (e.g., Informatica, Talend, Databricks, AWS Glue, or similar).- Strong knowledge of SQL and experience with relational databases (e.g., Oracle, SQL Server, PostgreSQL, MySQL).- Experience with cloud platforms (AWS, Azure, or GCP) and cloud-native data services.- Familiarity with data warehousing concepts and modern architectures (Snowflake, Redshift, BigQuery).- Understanding of data governance, data quality, and security best practices.- Excellent problem-solving, communication, and collaboration skills.Good to Have :- Experience with streaming platforms (Kafka, Kinesis, Spark Streaming).- Knowledge of containerization and orchestration (Docker, Kubernetes, Airflow).- Exposure to machine learning pipelines and data science collaboration.- Certifications in AWS/Azure/GCP data services (ref:hirist.tech)

  • EverestDX - Enterprise Data Architect - ETL

    4 weeks ago

    Hyderabad, India EverestDX Full time

    Skills and Development :- We are seeking a highly skilled and innovative Database Architect, with a special focus on PostgreSQL and Solr, to join our team.- The ideal candidate will have a deep understanding of database architecture, experience with ETL tools, and expertise in designing and optimizing data pipelines.- This role requires a quick learner who...

  • EverestDX - Technical Data Analyst

    4 weeks ago

    Hyderabad, India EverestDX Full time

    Job Description :Key Responsibilities :- Collaborate with business stakeholders to gather requirements, define KPIs, and translate business needs into data solutions.- Design, develop, and optimize complex SQL queries, stored procedures, views, and database schemas to support reporting and analytical needs.- Develop and maintain ETL/ELT pipelines for data...

  • EverestDX - Technical Data Analyst

    4 days ago

    Hyderabad, Telangana, India EverestDX Inc Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job DescriptionKey Responsibilities :Collaborate with business stakeholders to gather requirements, define KPIs, and translate business needs into data solutions.Design, develop, and optimize complex SQL queries, stored procedures, views, and database schemas to support reporting and analytical needs.Develop and maintain ETL/ELT pipelines for data...

  • Generative AI Engineer

    6 days ago

    Hyderabad, Telangana, India EverestDX Inc Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About the Company:EverestDX Inc, a Product and Services company headquartered in Stamford, with global delivery centers in Hyderabad and Chennai . Our Platform/Solution includes Orchestration, Intelligent operations with BOTs', AI-powered analytics for Enterprise IT. Our vision is to enable Digital Transformation for enterprises to deliver seamless customer...

  • Java Spark Etl

    7 days ago

    Hyderabad, Telangana, India CALTEK Solutions Inc Full time

    Senior Java / Spark Developer. Required Skills & Experience: 8 Years of Java Experience Strong Spring boot experience Experience using Java-spark writing complex SQL, PL/SQL queries around Data Analysis, Data Lineage & Reconciliation preferably in PostgreSQL/Oracle Experience of creating Data Lineage documents, Source to Target (STM) Mapping documents and...

  • Data Engineer

    6 days ago

    Hyderabad, India Siyaton Software Solutions Pvt Ltd Full time

    Role : Data EngineerExperience : 6+ yearsLocation : Pune/HyderabadWork Mode : WFOJob Profile and proactive Data Engineer to lead and manage critical data infrastructure that powers our products and analytics. You will design and scale high-performance data pipelines and models across both our Data Lake and Data Warehouse environments. Responsible for...

  • EverestDX - Azure Databricks Administrator - Scala/Python

    6 days ago

    Hyderabad, India EverestDX Full time

    We are seeking a Cloud Databricks Administrator to join our Cloud Engineering team. In this role, you will monitor and maintain Databricks jobs, troubleshoot issues, ensure platform reliability, and manage access controls. You will play a key part in operational support, defect management, and proactive issue resolution to keep our data pipelines running...

  • Data Engineer

    2 weeks ago

    Hyderabad, India BOHIYAANAM TECHNOLOGY LLP Full time

    Description :Key Responsibilities :- Design, develop, and deploy end-to-end data engineering solutions using Databricks, Apache Spark, PySpark, Python, and SQL.- Build scalable and efficient ETL/ELT pipelines for data ingestion, transformation, and integration from various sources.- Work with data warehousing solutions and ensure high-performance data...

  • Lead/Sr. ETL Data Engineer

    4 weeks ago

    Hyderabad, India Data Economy Full time

    We are seeking a skilled ETL Data Engineer to design, build, and maintain efficient and reliable ETL pipelines, ensuring seamless data integration, transformation, and delivery to support business intelligence and analytics. The ideal candidate should have hands-on experience with ETL tools like Talend, strong database knowledge, and familiarity with AWS...

  • Lead engineer/data engineer

    7 days ago

    Hyderabad, India EverestDX Inc Full time

    Everest DX – We are a Digital Platform Services company, headquartered in Stamford. Our Platform/Solution includes Orchestration, Intelligent operations with BOTs’, AI-powered analytics for Enterprise IT. Our vision is to enable Digital Transformation for enterprises to deliver seamless customer experience, business efficiency and actionable insights...