Lead Big Data Engineer
4 days ago
What You'll Be Doing : - Build highly scalable, available, fault-tolerant distributed data processing systems (batch and streaming systems) processing over 100s of terabytes of data ingested every day and petabyte-sized data warehouse and elasticsearch cluster. - Build quality data solutions and refine existing diverse datasets to simplified models encouraging self-service. - Build data pipelines that optimize on data quality and are resilient to poor-quality data sources. - Own the data mapping, business logic, transformations, and data quality. - Low-level systems debugging, performance measurement & optimization on large production clusters. - Participate in architecture discussions, influence product roadmap, and take ownership and responsibility over new projects. - Maintain and support existing platforms and evolve to newer technology stacks and architectures. Ideal Candidate : - Proficiency in Python and PySpark. - Deep understanding of Apache Spark, Spark tuning, creating RDDs, and building data frames. - Experience in big data technologies like HDFS, YARN, Map-Reduce, Hive, Kafka, Spark, Airflow, Presto, etc. - Experience in building distributed environments using any of Kafka, Spark, Hive, Hadoop, etc. - Good understanding of the architecture and functioning of distributed database systems. - Experience working with various file formats like Parquet, Avro, etc., for large volumes of data. - Experience with one or more NoSQL databases. - Experience with AWS, GCP. - 5 years of professional experience as a data or software engineer. (ref:hirist.tech)
-
Lead Big Data Engineer
2 weeks ago
Bengaluru, Karnataka, India Andor Tech Full timeAndor Tech is Hiring: Lead Big Data Engineer Andor Tech is looking for a highly skilled Lead Big Data Engineer to join our team for deployment to a Retail MNC client in Bangalore . Job Details: Role: Lead Big Data Engineer Location: Bangalore Experience: Minimum 5+ Years Relevant Skills Experience: Scala: 3-4 Years SQL & PySpark: 3-4 Years Employment Type:...
-
Lead Big Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Andor Tech Full timeAndor Tech is Hiring: Lead Big Data EngineerAndor Tech is looking for a highly skilled Lead Big Data Engineer to join our team for deployment to a Retail MNC client in Bangalore .Job Details:Role: Lead Big Data EngineerLocation: BangaloreExperience: Minimum 5+ YearsRelevant Skills Experience:Scala: 3-4 YearsSQL & PySpark: 3-4 YearsEmployment Type: ...
-
Lead Big Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Andor Tech Full timeAndor Tech is Hiring: Lead Big Data Engineer Andor Tech is looking for a highly skilled Lead Big Data Engineer to join our team for deployment to a Retail MNC client in Bangalore.Job Details: Role: Lead Big Data Engineer Location: Bangalore Experience: Minimum 5+ Years Relevant Skills Experience:Scala: 3-4 YearsSQL & PySpark: 3-4 Years Employment...
-
Big Data Engineer Lead
4 days ago
Bengaluru, Karnataka, India Talent Worx Full timeJob Description:PYSPARK ARCHITECT AND MANAGERA highly skilled PySpark Architect and Manager is required to lead our data engineering initiatives. The ideal candidate will have deep expertise in PySpark, Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be responsible for designing, building, and managing large-scale data processing...
-
Lead Big Data Developer
11 hours ago
Bengaluru, Karnataka, India EXL Service Full timeAbout the PositionWe are seeking a highly experienced Senior Manager to lead our Data Engineering team at EXL Service. The ideal candidate will have a strong background in data warehousing, big data engineering, and distributed systems.Main ResponsibilitiesData Warehousing: Design and develop data warehouses using various data warehousing concepts.Big Data...
-
Big Data Engineer
4 weeks ago
Bengaluru, Karnataka, India SMARTWORK IT SERVICES Full timeJob Title : Big Data Engineer (Dynatrace)Location : Chennai, Bangalore, HyderabadExperience : 6-10 YearsRequired skills : Big Data, DynatraceJob Description : We are looking for an experienced Big Data Engineer (Dynatrace) with building scalable data pipelines and ensuring performance optimization through Dynatrace monitoring. This role requires a seasoned...
-
Lead Big Data Software Engineer
3 weeks ago
Bengaluru, Karnataka, India Epam Full timeDescription EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that...
-
Lead Big Data Developer
8 hours ago
Bengaluru, Karnataka, India Yogy HR Solutions Full timeAbout the RoleYogy HR Solutions is seeking a highly skilled Lead Big Data Engineer to join our team. In this role, you will be responsible for designing, developing, and testing big data solutions using Spark, Scala, and SQL.">Job DescriptionWe are looking for an experienced big data developer with hands-on experience in Spark, Scala, and SQL. The ideal...
-
Big Data Engineer
4 days ago
Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full timeJob Title : Big Data Engineer Location : Bangalore Employment Type : Full-time Work Mode : 4 Days Work from Office Notice Period : Immediate to 30 days Job Description : We are looking for a Big Data Engineer with expertise in Hadoop, Hive, SQL, Python, PySpark, and AWS to join our team in Bangalore. The ideal candidate will have hands-on experience in...
-
Big Data Analytics Engineer
3 days ago
Bengaluru, Karnataka, India Skilltasy Full timeJob Description: We are seeking a highly skilled Big Data Analytics Engineer to join our team. As a Big Data Analytics Engineer, you will design, develop, and implement big data analytics solutions using various tools and technologies. Main Responsibilities:Design and develop big data analytics solutions using various tools and technologies.Analyze large...
-
Big Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Infogain Full timeBig Data Engineer (Lead) with skills Data Engineering, Data Warehouse, Big Data, snowflake, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune) Posted on: July 30, Share on Linkedin Share on Twitter Share on Facebook ROLES & RESPONSIBILITIES Responsibilities: · Strong experience of minimum 5+...
-
Big Data Engineer
7 days ago
Bengaluru, Karnataka, India Scoop Technologies Pvt Ltd Full timeJob Title : Big Data EngineerLocation : BangaloreEmployment Type : Full-timeWork Mode : 4 Days Work from OfficeNotice Period : Immediate to 30 daysJob Description :We are looking for a Big Data Engineer with expertise in Hadoop, Hive, SQL, Python, PySpark, and AWS to join our team in Bangalore. The ideal candidate will have hands-on experience in designing,...
-
Big Data Engineer
3 days ago
Bengaluru, Karnataka, India Tata Consultancy Services Full timeWelcome to Tata Consultancy Services, where we are driven by innovation and excellence. We are seeking an experienced Big Data Engineer to join our team.As a Big Data Engineer, you will work on designing and implementing scalable ETL/ELT processes and modeling data for low latency reporting. You will also work with large cloud data lakes and have experience...
-
Big Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Human Horizon Full timeResponsibilities :- Design, develop, and execute test cases for large-scale big data platforms.- Perform testing on various Big Data technologies (e. g., Hadoop, Hive, Spark, NoSQL databases).- Ensure data accuracy, integrity, and quality in data processing and transformations.- Work closely with developers, data engineers, and business analysts to...
-
Big Data Engineer
2 weeks ago
Bengaluru, Karnataka, India Human Horizon Full timeResponsibilities :- Design, develop, and execute test cases for large-scale big data platforms.- Perform testing on various Big Data technologies (e. g., Hadoop, Hive, Spark, NoSQL databases).- Ensure data accuracy, integrity, and quality in data processing and transformations.- Work closely with developers, data engineers, and business analysts to...
-
Big Data Engineer
3 days ago
Bengaluru, Karnataka, India Vishanz Business Services Private Limited Full timeVishanz Business Services Private Limited is seeking an experienced Big Data Engineer to design and develop scalable data solutions. As a Data Engineer, you will work on creating data pipelines and ETL processes that handle large volumes of data.Key TasksDesign and develop big data solutions using Microsoft Azure services.Collaborate with data scientists and...
-
Big Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Telescopp Full timeJob DescriptionJob OverviewWe are looking for a talented Big Data Engineer to join our team in Bengaluru. This is a mid-level, full-time position requiring a minimum of 4 years to a maximum of 6 years of relevant work experience. As a Big Data Engineer at Telescopp, you will be responsible for developing, testing, and maintaining big data solutions to...
-
Big Data Engineer Lead
16 hours ago
Bengaluru, Karnataka, India Amazon Full timeKey Responsibilities: As a Data Engineer on our DISCO team, you will design and develop big data pipelines, build data models, and deploy data sets using AWS technologies like Redshift, S3, EMR, EC2, DynamoDB, Kinesis Firehose, and Lambda. You will also partner with Product, Marketing, BI, and ML teams to build new behavioral events, pipelines, datasets,...
-
Big Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Telescopp Full timeJob OverviewWe are looking for a talented Big Data Engineer to join our team in Bengaluru. This is a mid-level, full-time position requiring a minimum of 4 years to a maximum of 6 years of relevant work experience. As a Big Data Engineer at Telescopp, you will be responsible for developing, testing, and maintaining big data solutions to support our client's...
-
Big Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Adobe Full timeOur Company Changing the world through digital experiences is what Adobe's all about. We give everyone—from emerging artists to global brands—everything they need to design and deliver exceptional digital experiences We're passionate about empowering people to create beautiful and powerful images, videos, and apps, and transform how companies...