Current jobs related to Big Data Engineer - Bengaluru, Karnataka - Thoucentric
-
Data Engineer
2 months ago
Bengaluru, Karnataka, India Talent500 Tech Full timeWe are Talent500:We help companies hire, build and manage global teams. We are trusted by the worlds leading companies - from Fortune 500s and larger enterprises (Walmart, Target, Nike, Pepsi etc) to industry leaders (Airbnb, Twitter, Uber) and some of the worlds fastest-growing start-ups from the portfolios of investors such as Accel, Sequoia and...
-
Staff Data Engineer
2 months ago
Bengaluru, Karnataka, India Leading-Biotechnology-Company Full timeWe are seeking a highly experienced Staff Data Engineer with 10+ years of experience to join our talented team in Bangalore. This position will lead our data engineering efforts in Bangalore. In this strategic role, you will be responsible for designing, implementing, and optimizing complex data solutions using Python, Spark, SQL, Snowflake, dbt, machine...
-
Lead Engineer
3 months ago
Bengaluru, Karnataka, India Kenvue Full timeThe Data Platform Administratoris responsible for leading software design, service architecture, and software development/solution engineering. You will manage junior engineers, lead design and code reviews within the service, and investigate emerging technologies to drive continuous improvement of the service:Key Responsibilities:Administer, configure, and...
-
Data Engineer
1 week ago
Bengaluru, Karnataka, India globalinsurance Full timeTechnology at Brit:Brit has a leadership position at Lloyds, the most well-known insurance market in the world. At Brit, we believe that the uncertainty of the future should never stand in the way of progress. Thats why we exist. To help people and businesses face the future and thrive.We are change-makers enabled by a global workforce who collaborate to...
-
Data Engineer
4 months ago
Bengaluru, Karnataka, India arko Full timeJob Description:We are seeking a skilled Data Engineer with 5-8 years of experience to join our team. The ideal candidate will have expertise in Power BI data modelling, master data management, data warehousing, and Azure data platform technologies. If you have a passion for transforming data into actionable insights and driving business decisions, we want...
-
Lead Data Engineer
1 month ago
Bengaluru, Karnataka, India Delta Air Lines Full timeAbout Delta Tech Hub:Delta Air Lines (NYSE: DAL) is the U.S. global airline leader in safety, innovation, reliability and customer experience. Powered by our employees around the world, Delta has for a decade led the airline industry in operational excellence while maintaining our reputation for award-winning customer service. With our mission of connecting...
-
Developer - Data Engineering
1 month ago
Bengaluru, Karnataka, India 3M Technology Center of Excellence Full timeDeveloper - Data Engineering is a part of the Data & Analytics function to work on designing, developing, testing, and supporting Digital Data Core pipelines and complex data models development in Amazon Web Services (AWS) and deliver insights to the business.Primary Responsibilitiesinclude but are not limited to the following:Designs, develops, and...
-
Staff Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Leading-Biotechnology-Company Full timeJOB DUTIES:Translate business requirements into data requirements, data warehouse design and sustaining data management strategies on enterprise data platforms like Snowflake.Work with project leads, stakeholders, and business SMEs to define technical specifications to develop data modeling requirements and maintain data infrastructure to provide business...
-
Lead Data Engineer
3 months ago
Bengaluru, Karnataka, India Delta Air Lines Full timeKey Responsibilities:Create, construct, and manage data pipelines with AWS services including S3, Glue, EMR, and Redshift.Use Kinesis, Stream, and Firehose to implement batch and stream data loading.For infrastructure as code, use AWS CDK to ensure scalability and dependability.Improve the effectiveness and performance of SQL queries.Write Python and other...
-
Finance Data Engineer
1 month ago
Bengaluru, Karnataka, India 3M Technology Center of Excellence Full timeFinance Data EngineerFinance Data Engineer will be responsible for implementing data intelligence solutions around Snowflake Data Warehouse for global finance organization.Responsibilities:Analyzes, designs, develops, tests, and supports data analytical solutions withinSnowflake Cloud Data WarehouseDeveloping ETL pipelines in and out of data warehouse using...
-
Lead Engineer
3 months ago
Bengaluru, Karnataka, India Kenvue Full timeThe CPD Data Engineering Architect is responsible for analyzing and designing technology solutions to meet business needs.Key Responsibilities:Design and implement scalable and efficient data pipelines to support business needsCollaborate with cross-functional teams to identify and solve complex data-related problemsDevelop and maintain data architecture and...
-
Sr Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Leading-Biotechnology-Company Full timeSenior Data EngineerJOB DUTIES:Design and architect data lake solutions on AWS S3 and Snowflake, considering scalability, automation, security, and performance requirements.Responsible for architecting, building, and optimizing data lake to store, process and analyze large volumes of structure and unstructured data.Requirements gathering and analysis,...
-
Sr Engineer
3 months ago
Bengaluru, Karnataka, India Kenvue Full timeThe Data Integration Senior Software Engineer is responsible for leading IICS, SAP Data Services Datasphere & ADF related build & Support projects.Key Responsibilities:Lead the design, implementation, and optimization of automated build, deployment, and release processes for data integration solutions across IICS, SAP Data Services, Datasphere and ADF.Ensure...
-
Engineer II
1 month ago
Bengaluru, Karnataka, India lululemon India Tech Hub Full timeAbout lululemon:lululemon is an innovative performance apparel company for yoga, running, training, and other athletic pursuits. Setting the bar in technical fabrics and functional design, we create transformational products and experiences that support people in moving, growing, connecting, and being well. We owe our success to our innovative products,...
-
Engineer II
2 months ago
Bengaluru, Karnataka, India lululemon India Tech Hub Full timeAbout lululemon:lululemon is an innovative performance apparel company for yoga, running, training, and other athletic pursuits. Setting the bar in technical fabrics and functional design, we create transformational products and experiences that support people in moving, growing, connecting, and being well. We owe our success to our innovative products,...
-
Senior Software Engineer
2 weeks ago
Bengaluru, Karnataka, India BBY India Full timeKey ResponsibilitiesLead the delivery of complex solutions, coding larger features from start to finishActively participate in planning, and performing code and architecture reviews of their teams productHelp ensure the quality and integrity of the SDLC for their team and identify opportunities for improvement in how the team works, through usage of...
-
Senior Software Development Engineer
1 week ago
Bengaluru, Karnataka, India Delta Air Lines Full timeAbout Delta Tech Hub:Delta Air Lines (NYSE: DAL) is the U.S. global airline leader in safety, innovation, reliability and customer experience. Powered by our employees around the world, Delta has for a decade led the airline industry in operational excellence while maintaining our reputation for award-winning customer service. With our mission of connecting...
-
Staff Engineer
4 weeks ago
Bengaluru, Karnataka, Karnataka, India GE Aviation Full timeJob Description SummaryThe Staff Engineer demonstrates leadership for functional, business, and broad company objectives. In this role you will provide strategic, technical and project leadership in a cross functional team developing analytic products for aircraft engine components. You will have exposure to large datasets, big data pattern recognition,...
-
Data Steward
2 months ago
Bengaluru, Karnataka, India Leading-Biotechnology-Company Full timePosition Summary:Executes EAM practices and procedures with while providing expert input on best practices for calibration program, procedures, frequencies and tolerances. They will support the day to day operations of engineering systems and processes.Supports and completes EAM Data Steward responsibilities for all functions using EAM and for all functional...
-
Software Engineer II
1 month ago
Bengaluru, Karnataka, India BBY India Full timeKey ResponsibilitiesContribute to the delivery of complex solutions, breaking down big problems into smaller piecesActively participate in team planning activitiesEnsure the quality and integrity of the Software Development Life Cycle (SDLC) for their team and identify opportunities for improvement in how the team works, through usage of recommended tools...
Big Data Engineer
3 months ago
Job Description:
As a Big Data Engineer, you will be responsible for designing, developing, and maintaining our big data infrastructure. You will work with large datasets, perform data processing, and support various business functions by creating data pipelines, data processing jobs, and data integration solutions. You will be working in a dynamic and collaborative environment, leveraging your expertise in Hive, Hadoop, and PySpark to unlock valuable insights from our data.
Key Responsibilities:
- Data Ingestion and Integration:
Develop and maintain data ingestion processes to collect data from various sources.
Integrate data from different platforms and databases into a unified data lake.
- Data Processing:
Create data processing jobs using Hive and PySpark for large-scale data transformation.
Optimize data processing workflows to ensure efficiency and
- Pipeline Development:
Design and implement ETL pipelines to move data from raw to processed formats.
Monitor and troubleshoot data pipelines, ensuring data quality and
- Modeling and Optimization:
Develop data models for efficient querying and reporting using Hive.
Implement performance tuning and optimization strategies for Hadoop and
- Governance:
Implement data security and access controls to protect sensitive information.
Ensure compliance with data governance policies and best practices.
Bachelor's degree in Computer Science, Information Technology, or a related field.
3+ years of experience in big data engineering and data processing.
Proficiency in Hive, Hadoop, and PySpark.
Strong SQL and NoSQL database experience.
Experience with data warehousing and data modeling.
Knowledge of data integration, ETL processes, and data quality.
Strong problem-solving and troubleshooting skills.
Excellent communication and teamwork skills.
Preferred Qualifications:
Experience with cloud-based big data technologies (e.g., AWS EMR, Azure HDInsight, Google Dataprep).
Certification in Hadoop, Hive, or PySpark.
Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).
Knowledge of data visualization tools (e.g., Tableau, Power BI).