PhysicsWallah - Data Engineer - ETL

2 weeks ago

Bangalore, Karnataka, India PhysicsWallah Full time

Working Experience :

- Minimum 1 year of work experience.

- Startup experience preferred, Streaming Platform, Edtech work experience bonus.

Roles & Responsibilities :

- Develop ETL (Extract, Transform, Load) processes to move and transform data between systems.

- Integrate data from various sources, ensuring consistency and accuracy.

- Monitor system performance and identify areas for optimization.

- Align architecture with business requirements and provide solutions which fit best to solve the business problems.

- Build the infrastructure required for optimal ETL pipelines from a wide variety of data sources using SQL and AWS big data technologies.

- Evaluate and recommend new technologies to enhance data processing capabilities.

- Work closely with data scientists, analysts, and other stakeholders to understand their data needs.

- Use programming language( python, spark) and tools efficiently to collate the data.

- Identify ways to improve data reliability, efficiency and quality.

- Use data to discover tasks that can be automated.

Skill Sets :

- Can Develop ETL (Extract, Transform, Load) processes to move and transform data between systems.

- Have good hands-on experience in any coding language Python or Scala.

- Good understanding of Data Engineering concepts such as data warehousing, datalakes, data modeling, data quality assurance, data lifecycle management, streaming data, metadata management etc.

- Strong hold on cloud-based ecosystems(AWS preferred).

- Have an understanding of data pipeline orchestration tools such as Airflow or Azkaban (Airflow Preferable).

- Can work with spark, EMR and Kubernetes cluster.

- Can write spark jobs, shell scripts and optimized data pipelines.

- Experience with relational SQL and NoSQL databases, including Postgres, Amazon Redshift and MongoDB.

- Have an understanding of OOPS-based data platforms, and can develop data platform modules.

- Good understanding of backend APIs and their use cases.

(ref:hirist.tech)

  • PhysicsWallah - Regional Analyst - Python/Data Visualization

    2 months ago

    Bangalore, Karnataka, India PhysicsWallah Full time

    Job Description : Company Description : Physics Wallah is an Indian online education technology startup based in Delhi, originally created as a YouTube channel in 2014 by Mr. Alakh Pandey. We are the first company aiming to build an affordable online education platform for each Indian student who dreams of IIT & AIIMS but is unable to afford the existing...

  • PhysicsWallah - Regional Analyst - Python/Data Visualization

    2 weeks ago

    Bangalore, Karnataka, India PhysicsWallah Full time

    Job Description : Company Description : Physics Wallah is an Indian online education technology startup based in Delhi, originally created as a YouTube channel in 2014 by Mr. Alakh Pandey. We are the first company aiming to build an affordable online education platform for each Indian student who dreams of IIT & AIIMS but is unable to afford the existing...

  • Data Engineer

    2 months ago

    Bangalore, Karnataka, India Mobile Programming LLC Full time

    Job Description :Please note : This position is focused on development rather than migration. We are seeking an experienced Data Engineer to join our team. As a Data Engineer, you will play a crucial role in developing and maintaining our data infrastructure and ensuring the smooth operation of our data platforms. The ideal candidate should have a strong...

  • Data Engineer

    2 weeks ago

    Bangalore, Karnataka, India Mobile Programming LLC Full time

    Job Description :Please note : This position is focused on development rather than migration. We are seeking an experienced Data Engineer to join our team. As a Data Engineer, you will play a crucial role in developing and maintaining our data infrastructure and ensuring the smooth operation of our data platforms. The ideal candidate should have a strong...

  • PhysicsWallah - Backend Developer - Node.js

    2 weeks ago

    Bangalore, Karnataka, India PhysicsWallah Full time

    Role: Backend Developer (SDE 2 and SDE 3) NodeJs. Location: Bangalore (WFO). Responsibilities:.- Work closely with 2-4 engineers directly to drive development projects.- Work with EM and architects to develop solutions which are scalable and robust.- Mentor junior engineers in the team to drive excellence. Requirements :- 4+ years of experience in Tech...

  • PhysicsWallah - Backend Developer - Node.js

    2 months ago

    Bangalore, Karnataka, India PhysicsWallah Full time

    Role: Backend Developer (SDE 2 and SDE 3) NodeJs. Location: Bangalore (WFO). Responsibilities:.- Work closely with 2-4 engineers directly to drive development projects.- Work with EM and architects to develop solutions which are scalable and robust.- Mentor junior engineers in the team to drive excellence. Requirements :- 4+ years of experience in Tech...

  • Data Engineer

    2 weeks ago

    Bangalore, Karnataka, India CYBERSAPIENT TECHNOLOGIES PRIVATE LIMITED Full time

    Company Description : CST Cyber Sapient is a company dedicated to helping clients achieve advocacy through exceptional product and service experiences. We work closely with our clients to drive change within their organization and provide support in areas such as strategic vision, product innovation, user experience, talent support, and technical delivery....

  • Data Engineer

    2 months ago

    Bangalore, Karnataka, India CYBERSAPIENT TECHNOLOGIES PRIVATE LIMITED Full time

    Company Description : CST Cyber Sapient is a company dedicated to helping clients achieve advocacy through exceptional product and service experiences. We work closely with our clients to drive change within their organization and provide support in areas such as strategic vision, product innovation, user experience, talent support, and technical delivery....

  • Data Engineer

    30 minutes ago

    Bangalore, Karnataka, India ANKYAH INFINITY PRIVATE LIMITED Full time

    Job Description :- Design, develop, and implement ETL processes to extract, transform, and load data into data lakes or warehouses.- Ensure the accuracy, completeness, and timeliness of data transformations and integrations.- Collaborate with stakeholders to understand and translate business requirements into technical solutions.- Optimize ETL workflows for...

  • Data Engineer

    1 month ago

    Bangalore, Karnataka, India Careerzgraph Full time

    Total experience required : 15-20 years.Notice Period - Not More then 15 Days / Only Immediate Joiners For Location - Bengaluru (KA)/Electronic City Phase 1. - Work from Office Job Description :- Lead and mentor a team of data engineers in designing, developing, and maintaining efficient and reliable ETL pipelines.- Develop advanced scripts using Python for...

  • Data Engineer

    2 weeks ago

    Bangalore, Karnataka, India Careerzgraph Full time

    Total experience required : 15-20 years.Notice Period - Not More then 15 Days / Only Immediate Joiners For Location - Bengaluru (KA)/Electronic City Phase 1. - Work from Office Job Description :- Lead and mentor a team of data engineers in designing, developing, and maintaining efficient and reliable ETL pipelines.- Develop advanced scripts using Python for...

  • Senior Data Engineer

    2 months ago

    Bangalore, Karnataka, India LOGIC PLANET IT SERVICES Full time

    About the job :Project Role : Data EngineerMust have skills : Google BigQueryMinimum 5 year(s) of experience is requiredAs a Data Engineer, you will be responsible for designing, developing, and maintaining data solutions for data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and...

  • Senior Data Engineer

    2 weeks ago

    Bangalore, Karnataka, India LOGIC PLANET IT SERVICES Full time

    About the job :Project Role : Data EngineerMust have skills : Google BigQueryMinimum 5 year(s) of experience is requiredAs a Data Engineer, you will be responsible for designing, developing, and maintaining data solutions for data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and...

  • Senior Data Engineer

    2 months ago

    Bangalore, Karnataka, India Amicon Hub Full time

    Data Engineer role.Location : BangaloreNeed only Immediate Joiners and candidates from Product based/Start up Companies.Responsibilities :- Analyze and organize raw data - Build data systems and pipelines- Evaluate business needs and objectives- Interpret trends and patterns- Conduct complex data analysis and report on results - Prepare data for prescriptive...

  • Senior Data Engineer

    2 weeks ago

    Bangalore, Karnataka, India Amicon Hub Full time

    Data Engineer role.Location : BangaloreNeed only Immediate Joiners and candidates from Product based/Start up Companies.Responsibilities :- Analyze and organize raw data - Build data systems and pipelines- Evaluate business needs and objectives- Interpret trends and patterns- Conduct complex data analysis and report on results - Prepare data for prescriptive...

  • PhysicsWallah - Senior Product Designer/Designer Lead

    1 month ago

    Bangalore, Karnataka, India PhysicsWallah Full time

    Position Overview : As a GPM / Director of Product Management : - You will own a suite of products end to end at PW (starting from ideating the idea to Product Solution to Product Development to Product adoption).- Play a critical role in shaping the PMF, product strategy & roadmap.- Lead a team of talented product managers.- Collaborate closely with cross...

  • PhysicsWallah - Senior Product Designer/Designer Lead

    2 weeks ago

    Bangalore, Karnataka, India PhysicsWallah Full time

    Position Overview : As a GPM / Director of Product Management : - You will own a suite of products end to end at PW (starting from ideating the idea to Product Solution to Product Development to Product adoption).- Play a critical role in shaping the PMF, product strategy & roadmap.- Lead a team of talented product managers.- Collaborate closely with cross...

  • Senior ETL/Data Engineer

    2 weeks ago

    Bangalore, Karnataka, India Overture Rede Private Limited. Full time

    About the Role :We are seeking a highly skilled and experienced Sr. ETL/Data Engineer to join our team and play a critical role in our data warehousing and data management processes.In this role, you will be responsible for designing, developing, implementing, and maintaining ETL (Extract, Transform, Load) pipelines to deliver high-quality data to our data...

  • Senior Data Engineer

    2 weeks ago

    Bangalore, Karnataka, India D Square Consulting Services Pvt Ltd Full time

    Role : Sr. Data Engineer (DOMO)This is a full-time position with D Square Consulting Services Pvt LtdExperience - 5+ YearsLocation - BangaloreCandidates who can join within 30 days, may only applyWorking Mode: Work From OfficeJob Highlights:Must-Have skills :- 5+ years' experience in ELT systems (Snowflake and DOMO)- Must have experience in digital...

  • Senior Data Engineer

    2 weeks ago

    Bangalore, Karnataka, India D Square Consulting Services Pvt Ltd Full time

    Role : Sr. Data Engineer (DOMO)This is a full-time position with D Square Consulting Services Pvt LtdExperience - 5+ YearsLocation - BangaloreCandidates who can join within 30 days, may only applyWorking Mode: Work From OfficeJob Highlights:Must-Have skills :- 5+ years' experience in ELT systems (Snowflake and DOMO)- Must have experience in digital...