Senior Data Engineer

4 weeks ago


BangaloreAny Location, IN SWAI TECHNOLOGIES PRIVATE LIMITED Full time

Job Description :


We are seeking a highly skilled Senior Data Engineer with 5 - 12 years of experience in building scalable data solutions. The ideal candidate should be proficient in Python programming and SQL, with extensive experience in utilizing various AWS services for data processing and storage.


Experience with big data processing environments, orchestration tools, and ETL processes is essential for this role.


Primary Skills :

  • Programming Languages: Strong proficiency in Python for data manipulation, transformation, and analysis.
  • Database: Proficient in SQL for querying and managing relational databases.
  • AWS Services: Extensive experience with AWS services such as S3, Redshift, EMR, Glue, Kinesis, DynamoDB, Athena, AWS Lambda, Step Functions, Lake Formation, Glue Workflow, Event Bridge, and Redshift Spectrum for building scalable data solutions on the cloud platform.
  • Big Data: Experience working in big data processing environments using tools like Apache Spark for distributed data processing and analytics.
  • Orchestration Tools: Familiarity with workflow orchestration tools like Apache Airflow, DBT (Data Build Tool), or similar services for managing and scheduling data pipelines.

Nice-to-Have Skills :

  • ETL Tools: Experience with ETL (Extract, Transform, Load) tools such as SSIS, Informatica, Talend, and Fivetran for data integration and processing.
  • Data Visualization: Familiarity with data visualization tools like QuickSight, Tableau, or Power BI for creating interactive and insightful visualizations of data.
  • Additional Technologies: Experience with additional data platforms like Databricks or Snowflake for advanced analytics and data warehousing.
  • Knowledge of CI/CD setup and Infrastructure-as-code practices is a plus.

Responsibilities :

  • Design, develop, and maintain scalable data pipelines and ETL processes to ingest, process, and transform data from various sources using Python and SQL.
  • Implement and optimize data processing workflows using AWS services like Glue, EMR, and Lambda functions for efficient data extraction, loading, and transformation.
  • Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver data solutions that meet business needs.
  • Ensure data quality and integrity by implementing data validation and monitoring processes throughout the data lifecycle.
  • Stay updated with emerging technologies and best practices in data engineering and cloud computing to continuously improve data processes and solutions.

Requirements :

  • Bachelor's degree in Computer Science, Engineering, or related field.
  • 5 - 12 years of experience in data engineering roles, with a focus on building scalable data solutions.
  • Strong proficiency in Python programming and SQL for data manipulation and analysis.
  • Hands-on experience with AWS cloud services and big data technologies like Spark.
  • Excellent problem-solving, analytical, and communication skills.
  • Ability to work independently and collaborate effectively in a team environment.
(ref:hirist.tech)

  • Any Location, IN K2 Data & Analytics Full time

    Job Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...


  • Any Location, IN K2 Data & Analytics Full time

    Job Description :Join our team in crafting the future of media with our cutting-edge platforms and technologies. We're on the lookout for highly motivated senior software engineers with a knack for building innovative software. If you're passionate about leveraging ML, search, and recommendation algorithms while mastering distributed architectures,...


  • Any Location/Bangalore, IN Harmony Data Integration Technologies Pvt. Ltd. Full time

    Job Description :We are seeking an experienced and proactive Senior DevOps Engineer with a strong background in Azure to join our dynamic team. The ideal candidate is not only a technical expert but also possesses excellent communication and problem-solving skills. As a Senior DevOps Engineer, you will work as support engineer and play a crucial role in...

  • Senior Data Engineer

    3 weeks ago


    Bangalore/Any Location, IN Randstad Full time

    Job Description:We are looking for a highly skilled Senior Data Engineer with over 5 years of experience in the data engineering field. The ideal candidate will have strong expertise in PySpark, SQL databases, and Big Data programming for data transformation and aggregations. This role offers an exciting opportunity to work on cutting-edge projects,...


  • Bangalore/Any Location, IN TalentXo Full time

    Criteria :Role : Strong Senior Data Warehouse/Data Engineer - Must have more than 8+ YOE in Software Development- Must have 6+ YOE in Data Ware house/Data Engineering and Data Modelling- Must have strong work experience in Any Dataware House / Data Lake- Must have Good Experience in SQL on any databases MySQL / PostgreSQL / Postgres / Oracle / SQL Server /...

  • Senior Data Engineer

    4 weeks ago


    Any Location, IN Muoro Full time

    Role & Responsibilities:- Design, architect, and develop code in a testable, scalable, and performant manner. - Work closely with Product Managers, Cross Product Engineers and Data Engineers to optimize and deliver innovative solutions. - Build a team that mines, analyzes, and transforms large [un]structured datasets, collaborating with analysts, data...


  • Any Location/Bangalore, IN Harmony Data Integration Technologies Pvt. Ltd. Full time

    Job Description :As a Senior Software Engineer, you will play a pivotal role in the design, development, and implementation of robust software solutions. Your key responsibilities will include, but not limited to,architecting scalable and maintainable systems, hands-on coding, crafting robust unit tests, conducting thorough code reviews, and collaborating...

  • MLOps Developer

    6 hours ago


    Any Location, IN GOOGNU DATA SERVICES PRIVATE LIMITED Full time

    Job Description :- Design and Implement MLOps Infrastructure: Build and maintain robust MLOps workflows using Azure Machine Learning and other Azure services to automate machine learning lifecycle from data preparation to model deployment and monitoring.- Collaborate with Cross-Functional Teams: Work closely with data scientists, software developers, and IT...

  • Data Engineer

    4 weeks ago


    Bangalore/Any Location, IN INSIGHT GLOBAL SOLUTIONS Full time

    Data Engineer Must Haves :- Bachelors Degree in science, technology, or another related field.- 3+ years of data engineering experience.- 3+ years of experience with Cloud platforms like GCP, Azure, Airflow.- 4+ years of data warehouse, relational database, and SQL experience.- 4+ years of supporting ETL process and job scheduling experience.- 3+ years of...


  • Any Location, IN Mobile Programming LLC Full time

    Job Description : Mobile Programming is urgently hiring for a Data visualization Engineer who is responsible for the graphical representation of information and data. To be successful in this field, you must have knowledge of visual elements like charts, graphs, and maps, data visualization tools provide an accessible way to see and understand trends,...

  • Senior Data Scientist

    4 weeks ago


    Any Location/Bangalore, IN NLB Services Full time

    Senior Data Scientist role for one of our product based company in Bangalore location.Notice Period : Immediate to 30Days. Location : Bangalore.Skills : Python, LLM, NLP and Problem solving. Company Description : Founded in 2007, NLB Services is a global technology and digital talent solutions provider. Headquartered in Alpharetta, Georgia (USA), we have a...

  • Senior Data Analyst

    2 weeks ago


    Any Location/Bangalore, IN TalentXo Full time

    The OpportunityWe're seeking a highly motivated Senior Data Analyst to join our dynamic Data Science team. In this role, you'll play a vital role in the entire data analytics lifecycle, from data acquisition and cleaning to analysis, visualization, and reporting. You'll collaborate closely with cross-functional teams to leverage data insights to...

  • Data Engineer

    1 week ago


    Any Location, IN Reliable Group Full time

    POSITION DESCRIPTION :WHAT YOU'LL DO :- Architect, Design, document & implement the data pipelines to feed data models for subsequent consumption in Snowflake using dbt, airflow.- Ensure correctness and completeness of the data being transformed via engineering pipelines for end consumption in Analytical Dashboards.- Actively monitor and triage technical...

  • Data Engineer

    4 weeks ago


    Any Location, IN Muoro Full time

    Responsibilities :- Design, develop, and maintain robust data pipelines using Apache Spark, Python, and other relevant technologies.- Implement data ingestion processes from various sources, ensuring data quality and integrity.- Optimize and tune data pipelines for performance and scalability.- Collaborate with data scientists and analysts to deploy machine...


  • Any Location, IN Primo Hiring Solutions Full time

    Role Description :This is a full-time remote role for a Senior Backend Engineer. As a Senior Backend Engineer, you will be responsible for designing, developing, and maintaining the server-side of web applications and APIs. You will collaborate with cross-functional teams to understand business requirements and translate them into scalable, reliable, and...

  • Azure Data Engineer

    3 weeks ago


    Any Location, IN Visionyle Full time

    Position Name: Azure Data EngineerPosition Mode: PermanentExperience: 6 YearsJoining Location: RemoteNotice: ImmediateMode of Interview : 2-3 roundsSUMMARY:Azure Data Engineer should have experience on Azure data lake, Azure SQL database, Databricks, Azure Data Factory, and Azure SQL data warehouse and controlling and granting database access and migrating o...

  • Azure Data Engineer

    4 weeks ago


    Any Location/Bangalore, IN Zyoin group Full time

    Skills :Proficiency in Azure Data EngineeringStrong expertise in SSIS (SQL Server Integration Services)Extensive experience in ETL (Extract, Transform, Load) processesAdvanced skills in Power BI for data visualization and analyticsResponsibilities :Design, develop, and implement data solutions leveraging Azure Data Engineering services.Utilize SSIS for...

  • Azure Data Engineer

    4 weeks ago


    Any Location/Bangalore, IN Diatoz Solutions Full time

    We Are Seeking A Skilled Azure Data Engineer To Join Our Dynamic Team. The Ideal Candidate Will Be Responsible For Designing, Developing, And Maintaining Data Solutions On The Microsoft Azure Platform. You Will Collaborate With Cross-Functional Teams To Gather Requirements, Design Data Pipelines, And Implement Solutions To Meet Business...

  • Data Engineer

    6 days ago


    Any Location/Bangalore, IN RS Consultants Full time

    About the job :Responsibilities :- Build, document, test and launch new data extract, transform and load (ETL) processes for one or more core data sets.- Collaborate with Business Intelligence team and other stakeholders for data requirements- Collaborate with the Data Management team to provide quality data to stakeholders.- Build data expertise and learn...


  • Any Location, IN RapidBraiins Full time

    Job Description :We are seeking a highly skilled and experienced Senior DevOps Site Reliability Engineer to join our dynamic team. The ideal candidate will have a proven track record of success in DevOps, Site Reliability Engineering (SRE), or development roles within SaaS-based or enterprise applications. As a Senior DevOps SRE Engineer, you will play a...