Data Engineer

2 weeks ago

Bengaluru, India Coders Brain Technology Private Limited Full time

Job Description : Data Engineer

- Role : Data Engineer

- Work Location : Bangalore (Work From Office)

- Experience : 3+ years

Job Overview :

We are looking for a proficient Data Engineer with expertise in Amazon Redshift, Python, Apache Airflow, dbt (Data Build Tool), API integration, and AWS. This role will be responsible for developing and maintaining scalable data pipelines, integrating data from multiple sources, and ensuring that our data architecture supports business intelligence, reporting, and analytics requirements. You will collaborate with cross-functional teams to build and optimize our data infrastructure and provide clean, high-quality data to the business.

Key Responsibilities :

- Data Pipeline Development : Build and maintain robust ETL/ELT pipelines using Python, Apache Airflow, and dbt to extract, transform, and load data from various sources into Amazon Redshift.

- Amazon Redshift Management : Design, optimize, and maintain Amazon Redshift clusters, ensuring the warehouse is capable of handling large-scale data efficiently.

- API Integration : Develop solutions to integrate external APIs for data ingestion, ensuring proper data extraction, transformation, and integration into our data infrastructure.

- Data Modeling : Create and maintain scalable data models in Redshift that support analytics and reporting needs, including designing star and snowflake schemas for optimized querying.

- AWS Infrastructure Management : Leverage AWS services such as S3, Lambda, EC2, and CloudWatch to build and maintain a scalable and cost-efficient data architecture.

- dbt (Data Build Tool) : Use dbt to manage and automate SQL transformations, ensuring modular, reusable, and well-documented data transformation logic.

- Workflow Orchestration : Utilize Apache Airflow to orchestrate and automate data workflows, ensuring reliable data pipelines and scheduled jobs.

- Data Quality & Testing : Implement and maintain data validation checks and testing frameworks to ensure data integrity, accuracy, and compliance across all data pipelines.

- Collaboration : Work closely with data scientists, analysts, and product teams to understand data needs and provide technical solutions that meet business objectives.

- Performance Optimization : Tune SQL queries and manage the performance of Redshift clusters to ensure fast, efficient data access and analysis.

- Data Governance : Enforce data governance policies to ensure compliance with security, privacy, and data quality standards throughout the data lifecycle.

Key Skills & Qualifications :

- Bachelors/Masters degree in Computer Science, Engineering, Data Science, or a related field.

- 3+ years of experience in data engineering with expertise in Amazon Redshift, Python, and AWS.

- Strong experience with Apache Airflow for workflow scheduling and orchestration.

- Hands-on experience with dbt (Data Build Tool) for managing SQL transformations and data models.

- Proficiency in API development and integration, including the use of RESTful APIs for data ingestion.

- Extensive experience with AWS services such as S3, Lambda, EC2, RDS, and CloudWatch.

- Expertise in data modeling concepts and designing efficient data structures (e.g., star schemas, snowflake schemas) in a data warehouse environment.

- Advanced knowledge of SQL for querying and optimizing large datasets in Redshift.

- Experience building ETL/ELT pipelines and integrating data from multiple sources, including structured and unstructured data.

- Familiarity with version control systems like Git and best practices for code management and deployment automation.

- Knowledge of data governance principles, including data security, privacy, and quality control.

Preferred Qualifications :

- Experience with real-time data processing tools such as Kafka or Kinesis.

- Familiarity with data visualization tools like Tableau, Looker, or Power BI.

- Knowledge of other data warehousing solutions like Snowflake or Google BigQuery.

- Experience with DevOps practices for managing infrastructure and CI/CD pipelines (Docker, Kubernetes).

- Understanding of machine learning pipelines and how data engineering supports AI/ML initiatives.

Soft Skills :

- Strong analytical and problem-solving skills.

- Ability to work independently and as part of a cross-functional team.

- Strong written and verbal communication skills, with the ability to explain technical concepts to non- technical stakeholders.

- Detail-oriented, proactive, and self-motivated with a focus on continuous improvement.

- Strong organizational and project management skills to handle multiple tasks in a fast-paced environment.

(ref:hirist.tech)

  • Data Engineer

    1 month ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 301314 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Job Duties: As a data...

  • Data Engineering Expert

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a skilled Data Engineer to join our team in Bangalore, India. As a key member of our data engineering team, you will play a crucial role in designing and implementing automated data pipelines.

  • Data Engineer

    1 month ago

    Bengaluru, India NTT DATA Full time

    Job Description Mandatory Skills and Qualifications:Snowflake: Proven experience in using Snowflake for data warehousing solutions.Databricks: Strong hands-on experience with Databricks for big data processing and analytics using Machine Learning.Redshift: Expertise in managing and optimizing Amazon Redshift data warehouses.Spark: Advanced knowledge of...

  • Data Engineering Cloud Specialist

    4 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data Engineering Cloud SpecialistAbout the RoleWe are seeking an experienced Data Engineer to join our team at NTT DATA. This is a fantastic opportunity to work on diverse data engineering activities on Azure Data platform and contribute to building a robust data pipeline connecting various enterprise data sources.About YouTo be successful in this...

  • Databricks Data Engineer

    1 month ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 301996 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Databricks Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Job Duties: Key...

  • Data Engineering Specialist

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: Data Engineering Specialist - BangaloreAbout the Role:We are seeking a highly skilled Data Engineering Specialist to join our team in Bangalore, India. As a key member of our engineering team, you will be responsible for designing and developing high-quality data pipelines using ETL code in Databricks.Key Responsibilities:Develop high-level design...

  • Cloud Data Engineer for Big Data Solutions

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the Role:As a Cloud Data Engineer, you will be responsible for designing and implementing large-scale data processing systems using GCP's BigQuery, Dataflow, and Dataproc technologies. Your expertise in ETL/ELT pipelines, data warehousing, and lakehouse solutions will enable us to efficiently process structured and unstructured data.Key...

  • Snowflake Data Engineer and Architect

    4 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Snowflake Data Engineer and Architect to join our team in Bangalore, India. This is a senior-level position that requires expertise in designing, developing, and maintaining large-scale data warehousing solutions using Snowflake.Key ResponsibilitiesDesign and implement efficient data models and architectures for...

  • AI Data Engineer

    3 weeks ago

    Bengaluru, India NTT DATA Full time

    Job Description Job Title: Data Science & AIML, GenAI Lead/EngineerKey Responsibilities:• Develop and implement traditional machine learning algorithms.• Deploy at least one model in a production environment.• Write and maintain Python code for data science and machine learning projects.Preferred Qualifications:• Knowledge of Deep Learning (DL)...

  • AI Data Science and Engineering Leader

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Title: AI Data Science and Engineering LeaderAbout the Role:We are seeking a highly skilled Ai Data Science and Engineering Leader to join our team in Bangalore, India. This is an excellent opportunity for a talented individual to lead our data science and engineering efforts.Responsibilities:Develop and implement traditional machine learning algorithms...

  • Sr. Data Engineer

    4 weeks ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 302055 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Sr. Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Job Duties: Build data...

  • Cloud Engineer

    3 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job OverviewNTT DATA, a global leader in business and technology services, is seeking a highly skilled Cloud Engineer - Data Specialist to join our team in Bangalore, India.About the RoleWe are looking for an experienced Cloud Engineer with expertise in designing, building, and maintaining scalable data pipelines using Databricks on AWS. The ideal candidate...

  • Hevo Data

    1 month ago

    Bengaluru, India HEVO DATA Full time

    This role provides a great opportunity to own and build highly scalable and fault-tolerant platform services that will help support thousands of customers' data pipelines and load data reliability to the different cloud warehouses. These services are the core of the Hevo platform that helps provide building no code-low pipeline. Also, makes a huge...

  • Data Engineer for Cloud Migration and Data Pipelining Expert

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Data Engineer to join our team in Bangalore, India. This role involves designing, building, and maintaining scalable data pipelines using Databricks on AWS.The ideal candidate will have experience with Databricks, including Apache Spark development, and proficiency in Python or Scala for Spark job development....

  • Power BI Data Engineer

    1 month ago

    Bengaluru, India NTT DATA Full time

    Job Description Req ID: 296408 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Power BI Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Job Duties: Must...

  • Cloud Data Engineer for Big Data Processing

    4 weeks ago

    Bengaluru, Karnataka, India NTT DATA Services Full time

    About the RoleWe are seeking a highly skilled Cloud Data Engineer to join our team in Bangalore, India. As a key member of our data engineering team, you will design, build, and maintain scalable data pipelines using Databricks on AWS.Job Responsibilities:Design and develop efficient data pipelines using Apache Spark and Databricks.Ingest data from multiple...

  • Lead Data Engineer

    4 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time

    Job Description:We are seeking a skilled Lead Data Engineer to join our team in Bangalore, India. As a key member of our data engineering team, you will be responsible for designing and implementing high-level design documents, building ETL code in Databricks, and performing unit and system integration testing.Responsibilities:Design and implement high-level...

  • Databricks Data Engineer

    2 months ago

    Bengaluru, India NTT DATA Services Full time

    Req ID: 301996 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Databricks Data Engineer to join our team in Bangalore, Karnātaka (IN-KA), India (IN).Job Duties: Key...

  • Solution Engineer

    2 months ago

    Bengaluru, India HEVO DATA Full time

    Overview :. We're seeking a talented Solutions Engineer with strong background in Data Domain to join our growing Sales team. You'll work one-on-one with the sales team to help discover our customers' business and technical needs and demonstrate how Hevo's platform can solve their real-time data replication needs. In this role, you will...

  • AI Data Engineer

    4 weeks ago

    Bengaluru, India NTT DATA Full time

    Job Description Job Duties: Must have: Microsoft Gen AI - copilot, Azure AI/MLHands on development of GenAI solutions using Microsoft Azure platform (Must have)Lead build team , maintain, deploy, and support the internal GenAI platform LLM /Langchain /OpenAI integration (API/SDK’s, services, platform backend, integration with third-party models,...