Senior Data Engineer

1 month ago

BangaloreAny Location, India SWAI TECHNOLOGIES PRIVATE LIMITED Full time

Job Description :


We are seeking a highly skilled Senior Data Engineer with 5 - 12 years of experience in building scalable data solutions. The ideal candidate should be proficient in Python programming and SQL, with extensive experience in utilizing various AWS services for data processing and storage.


Experience with big data processing environments, orchestration tools, and ETL processes is essential for this role.


Primary Skills :

  • Programming Languages: Strong proficiency in Python for data manipulation, transformation, and analysis.
  • Database: Proficient in SQL for querying and managing relational databases.
  • AWS Services: Extensive experience with AWS services such as S3, Redshift, EMR, Glue, Kinesis, DynamoDB, Athena, AWS Lambda, Step Functions, Lake Formation, Glue Workflow, Event Bridge, and Redshift Spectrum for building scalable data solutions on the cloud platform.
  • Big Data: Experience working in big data processing environments using tools like Apache Spark for distributed data processing and analytics.
  • Orchestration Tools: Familiarity with workflow orchestration tools like Apache Airflow, DBT (Data Build Tool), or similar services for managing and scheduling data pipelines.

Nice-to-Have Skills :

  • ETL Tools: Experience with ETL (Extract, Transform, Load) tools such as SSIS, Informatica, Talend, and Fivetran for data integration and processing.
  • Data Visualization: Familiarity with data visualization tools like QuickSight, Tableau, or Power BI for creating interactive and insightful visualizations of data.
  • Additional Technologies: Experience with additional data platforms like Databricks or Snowflake for advanced analytics and data warehousing.
  • Knowledge of CI/CD setup and Infrastructure-as-code practices is a plus.

Responsibilities :

  • Design, develop, and maintain scalable data pipelines and ETL processes to ingest, process, and transform data from various sources using Python and SQL.
  • Implement and optimize data processing workflows using AWS services like Glue, EMR, and Lambda functions for efficient data extraction, loading, and transformation.
  • Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver data solutions that meet business needs.
  • Ensure data quality and integrity by implementing data validation and monitoring processes throughout the data lifecycle.
  • Stay updated with emerging technologies and best practices in data engineering and cloud computing to continuously improve data processes and solutions.

Requirements :

  • Bachelor's degree in Computer Science, Engineering, or related field.
  • 5 - 12 years of experience in data engineering roles, with a focus on building scalable data solutions.
  • Strong proficiency in Python programming and SQL for data manipulation and analysis.
  • Hands-on experience with AWS cloud services and big data technologies like Spark.
  • Excellent problem-solving, analytical, and communication skills.
  • Ability to work independently and collaborate effectively in a team environment.
(ref:hirist.tech)

  • Senior Machine Learning Engineer

    1 month ago

    Any Location, India K2 Data & Analytics Full time

    Job Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...

  • Senior Machine Learning Engineer

    1 week ago

    Any Location, IN K2 Data & Analytics Full time

    Job Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...

  • Senior Machine Learning Engineer

    2 months ago

    Any Location, IN K2 Data & Analytics Full time

    Job Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...

  • Senior Machine Learning Engineer

    7 days ago

    Any Location, India K2 Data & Analytics Full time

    Job Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...

  • Senior Data Engineer

    7 days ago

    Any Location, IN Innova ESI Full time

    About the Role :We are seeking a highly skilled and experienced Senior Data Engineer to join our growing team. This is a remote-first position, offering you the flexibility to work from anywhere while collaborating effectively with colleagues. In this role, you will play a key role in the entire data lifecycle, from designing and implementing data pipelines...

  • Senior Data Engineer

    7 days ago

    Any Location, India Innova ESI Full time

    About the Role :We are seeking a highly skilled and experienced Senior Data Engineer to join our growing team. This is a remote-first position, offering you the flexibility to work from anywhere while collaborating effectively with colleagues. In this role, you will play a key role in the entire data lifecycle, from designing and implementing data pipelines...

  • Senior Mechanical Engineer

    1 month ago

    bangalore, India Abha Engineer Full time

    We are looking for a Senior Mechanical EngineerRoles are described below.1. Manpower Planning.2. Preparing of Project Cost.3. Schedule wise work execution.4. As Drawing & quality work execution.5. Client & Third Party Manage.6. Working Team Manage & Review.7. Reporting to Management.8. ROB & FOB Fabrication & Erection Work Knowledge.

  • Senior Mechanical Engineer

    7 days ago

    bangalore, India Abha Engineer Full time

    We are looking for a Senior Mechanical EngineerRoles are described below.1. Manpower Planning.2. Preparing of Project Cost.3. Schedule wise work execution.4. As Drawing & quality work execution.5. Client & Third Party Manage.6. Working Team Manage & Review.7. Reporting to Management.8. ROB & FOB Fabrication & Erection Work Knowledge.

  • SportsGameX - Senior Data Engineer - ETL

    4 hours ago

    Any Location, India SportsGameX Full time

    SportsGameX is seeking a highly skilled and experienced Senior Data Engineer to join our data team. This is an exciting opportunity to play a key role in designing and implementing robust data pipelines for our advanced analytics and machine learning applications.Position Overview : As a Senior Data Engineer, you will be responsible for architecting,...

  • Senior Backend Engineer

    1 month ago

    Any Location, India K2 Data & Analytics Full time

    Job Description : Join our team in crafting the future of media with our cutting-edge platforms and technologies. We're on the lookout for highly motivated senior software engineers with a knack for building innovative software. If you're passionate about leveraging ML, search, and recommendation algorithms while mastering distributed...

  • Senior Backend Engineer

    1 week ago

    Any Location, IN K2 Data & Analytics Full time

    Job Description :Join our team in crafting the future of media with our cutting-edge platforms and technologies. We're on the lookout for highly motivated senior software engineers with a knack for building innovative software. If you're passionate about leveraging ML, search, and recommendation algorithms while mastering distributed architectures,...

  • Senior Backend Engineer

    2 months ago

    Any Location, IN K2 Data & Analytics Full time

    Job Description :Join our team in crafting the future of media with our cutting-edge platforms and technologies. We're on the lookout for highly motivated senior software engineers with a knack for building innovative software. If you're passionate about leveraging ML, search, and recommendation algorithms while mastering distributed architectures,...

  • Senior Backend Engineer

    7 days ago

    Any Location, India K2 Data & Analytics Full time

    Job Description : Join our team in crafting the future of media with our cutting-edge platforms and technologies. We're on the lookout for highly motivated senior software engineers with a knack for building innovative software. If you're passionate about leveraging ML, search, and recommendation algorithms while mastering distributed...

  • Senior Data Engineer

    1 month ago

    Bangalore/Any Location, India Randstad Full time

    Job Description:We are looking for a highly skilled Senior Data Engineer with over 5 years of experience in the data engineering field. The ideal candidate will have strong expertise in PySpark, SQL databases, and Big Data programming for data transformation and aggregations. This role offers an exciting opportunity to work on cutting-edge projects,...

  • Senior Data Engineer

    1 week ago

    Bangalore/Any Location, IN Randstad Full time

    Job Description:We are looking for a highly skilled Senior Data Engineer with over 5 years of experience in the data engineering field. The ideal candidate will have strong expertise in PySpark, SQL databases, and Big Data programming for data transformation and aggregations. This role offers an exciting opportunity to work on cutting-edge projects,...

  • Senior Data Engineer

    2 months ago

    Bangalore/Any Location, IN Randstad Full time

    Job Description:We are looking for a highly skilled Senior Data Engineer with over 5 years of experience in the data engineering field. The ideal candidate will have strong expertise in PySpark, SQL databases, and Big Data programming for data transformation and aggregations. This role offers an exciting opportunity to work on cutting-edge projects,...

  • Senior Data Engineer

    7 days ago

    Bangalore/Any Location, India Randstad Full time

    Job Description:We are looking for a highly skilled Senior Data Engineer with over 5 years of experience in the data engineering field. The ideal candidate will have strong expertise in PySpark, SQL databases, and Big Data programming for data transformation and aggregations. This role offers an exciting opportunity to work on cutting-edge projects,...

  • Harmony Data Integration Technologies

    4 weeks ago

    Any Location/Bangalore, IN Harmony Data Integration Technologies Pvt. Ltd. Full time

    Job Description :We are seeking an experienced and proactive Senior DevOps Engineer with a strong background in Azure to join our dynamic team. The ideal candidate is not only a technical expert but also possesses excellent communication and problem-solving skills. As a Senior DevOps Engineer, you will work as support engineer and play a crucial role in...

  • Harmony Data Integration Technologies

    4 weeks ago

    Any Location/Bangalore, India Harmony Data Integration Technologies Pvt. Ltd. Full time

    Job Description : We are seeking an experienced and proactive Senior DevOps Engineer with a strong background in Azure to join our dynamic team. The ideal candidate is not only a technical expert but also possesses excellent communication and problem-solving skills. As a Senior DevOps Engineer, you will work as support engineer and play a crucial role in...

  • Senior Data Engineer

    4 weeks ago

    Chennai/Any Location, India mlesystems Full time

    Job Title: Senior Data Engineer - Airflow Location: Chennai (Remote) Company Overview: MLE Systems is a leading provider of advanced data analytics solutions, empowering businesses to harness the power of data for informed decision-making. We specialize in developing cloud-based platforms and cutting-edge technologies to drive actionable insights. We seek...