Data Engineer

2 months ago


india Expand My Business Full time

Details

Contract : 6 months to start with.

Experience 4-6yrs of experience

Location: Onsite (Bangalore)

Joining : Within 15 days.



Basic Qualifications

  • Bachelor's degree in Computer Science, Information Technology, or a related field.
  • Proven experience as a Data Engineer with expertise in ETL techniques (2+ years) is a MUST
  • Strong programming skills in languages such as Python, Java, or Scala.
  • Skillset to scrape and transform data off of the publicly available web sources
  • Experience with cloud-based data platforms (e.g., AWS, Azure, GCP).
  • Proficiency in SQL and experience working with relational and non-relational databases.
  • Knowledge of data warehousing concepts and architectures.
  • Familiarity with big data technologies such as Hadoop, Spark, and Kafka.
  • Experience with data modeling tools and techniques.
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration skills.


Preferred Qualifications

  • Master's degree or equivalent in Computer Science/ Data Science
  • Knowledge of data streaming and real-time processing.
  • Familiarity with data governance and security best practices.


Key Responsibilities:

  • ETL Development:
  • Design, develop, and maintain efficient ETL processes for multi-scale data sets.
  • Implement and optimize data transformation and validation processes to ensure data accuracy and consistency.
  • Collaborate with cross-functional teams to understand data requirements and business logic.
  • Data Pipeline Architecture:
  • Architect, build and maintain scalable and high-performance data pipelines
  • Evaluate and implement new technologies to enhance data pipeline efficiency and reliability
  • Pipelines for extracting data through scraping for ad-hoc sector specific datasets
  • Data Modelling:
  • Develop and implement data models to support analytics and reporting needs.
  • Optimize database structures for performance and scalability.
  • Data Quality and Governance:
  • Implement data quality checks and governance processes to ensure data integrity.
  • Collaborate with stakeholders to define and enforce data quality standards.
  • Documentation and Communication:
  • Document ETL processes, data models, and other relevant information.
  • Communicate complex technical concepts to non-technical stakeholders effectively.
  • Cross-functional collaboration:
  • Collaborate internally with the Quant team and developers to lay and optimize the data pipelines and externally with the stakeholders to understand the business requirements for the enrichment of the cloud database


  • india Data Warehouse Engineer Full time

    Experience : 2- 5 YearsPrimary Skills : Strong SQL knowledge, Data warehouse concepts and ETL hands on experience, Azure services like Data Factory, Azure Data Lake, and Good analysis skills.Good to have skills: Power BI, Databricks, Python.


  • india Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...


  • india Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement: Design and architect integration solutions to connect various enterprise applications, systems, and databases. Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications. Utilize Azure Integration Services such as Azure Logic Apps,...


  • India Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement: Design and architect integration solutions to connect various enterprise applications, systems, and databases. Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications. Utilize Azure Integration Services such as Azure Logic...

  • Lead data engineer

    1 week ago


    India Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, Py Spark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...

  • Lead Data Engineer

    2 weeks ago


    India Wavicle Data Solutions Full time

    Job Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...

  • Lead Data Engineer

    2 weeks ago


    India Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key...

  • Aws data engineer

    4 days ago


    India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on Py Spark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    3 months ago


    India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    3 months ago


    India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    3 weeks ago


    India ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    3 weeks ago


    india ITI Data Full time

    Job Description We are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources....

  • AWS Data Engineer

    3 months ago


    India ITI Data Full time

    Job DescriptionWe are looking for an AWS Data with primary skills on PySpark development who will be able to design and build solutions for one of our Fortune 500 Client programs, which aims towards building an Enterprise Data Lake on AWS Cloud platform, build Data pipelines by developing several AWS Data Integration, Engineering & Analytics resources. There...


  • India Wavicle Data Solutions Full time

    Wavicle Data Solutions is seeking an experienced Lead Data Engineer to play a pivotal role in architecting and implementing data solutions.About the RoleThis position offers a competitive salary of $180,000 - $220,000 per annum, commensurate with experience. Key ResponsibilitiesDesign and develop scalable and efficient data pipelines using Python and...


  • india Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key Responsibilities:...


  • india Wavicle Data Solutions Full time

    Job Description:We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering.As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions.Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success.Key Responsibilities:Lead...


  • india Wavicle Data Solutions Full time

    Job Description: We are seeking a highly experienced Lead Data Engineer with over 8 years of expertise in data engineering. As a Lead Data Engineer, you will play a pivotal role in architecting and implementing data solutions. Your proficiency in Python, PySpark, AWS, Databricks, SQL, and leadership skills will be crucial for success. Key Responsibilities:...

  • Data engineer

    1 month ago


    India Ryden Global Full time

    We're #hiring #Data Engineers for our USA based client at offshore in India (must support afternoon shift in IST).  Role Description This is a full-time/contract remote role for a Data Engineer at Ryden Global. The Data Engineer will be responsible for data engineering tasks such as data modeling, ETL processes, data warehousing, and data analytics...

  • Data Engineer

    1 month ago


    india Ryden Global Full time

    We're #hiring #DataEngineers for our USA based client at offshore in India (must support afternoon shift in IST). Role DescriptionThis is a full-time/contract remote role for a Data Engineer at Ryden Global. The Data Engineer will be responsible for data engineering tasks such as data modeling, ETL processes, data warehousing, and data analytics on a...

  • Data Engineer

    1 month ago


    India Ryden Global Full time

    We're #hiring #DataEngineers for our USA based client at offshore in India (must support afternoon shift in IST).  Role Description This is a full-time/contract remote role for a Data Engineer at Ryden Global. The Data Engineer will be responsible for data engineering tasks such as data modeling, ETL processes, data warehousing, and data analytics on a...