Highly Experienced Data Engineer
10 hours ago
About Talent21 Management and Shared Service Pvt Ltd
We are a leading organization in the industry, seeking an experienced Principal Software Engineer to join our Epsilon Attribution/Forecasting Product Development team.
Job Overview:
The successful candidate will be responsible for designing, implementing, and optimizing data processing solutions using Scala, Spark, and Hadoop.
Main Responsibilities:
- Design, implement, and optimize data processing solutions using Scala, Spark, and Hadoop
- Collaborate with cross-functional teams to deploy big data solutions on our on-premises and cloud infrastructure
- Build, schedule, and maintain workflows
- Perform data integration and transformation, troubleshoot issues, document processes, and communicate technical concepts clearly
- Continuously enhance our attribution engine/forecasting engine
Requirements:
To succeed in this role, you must have:
- Strong experience (12+ years) in Scala programming language
- Extensive experience with Apache Spark for Big Data processing
- Proficiency in performance tuning of Spark jobs, optimizing resource usage, shuffling, partitioning, and caching for maximum efficiency in Big Data environments
- In-depth understanding of the Hadoop ecosystem, including HDFS, YARN, and MapReduce
- Expertise in designing and implementing scalable, fault-tolerant data pipelines with end-to-end monitoring and alerting
- Using Python to develop infrastructure modules
- Solid grasp of database systems and SQLs for writing efficient SQLs (RDBMS/Warehouse) to handle TBS of data
- Familiarity with design patterns and best practices for efficient data modelling, partitioning strategies, and sharding for distributed systems
- End-to-end ownership with definition, development, and documentation of software objectives, business requirements, deliverables, and specifications in collaboration with stakeholders
- Experience in working on GIT (or equivalent source control) and solid understanding of Unit and integration test frameworks
- Able to collaborate with stakeholders/teams to understand requirements and develop a working solution
- Able to work within tight deadlines and effectively prioritize and execute tasks in a high-pressure environment
- Able to mentor junior staff
Preferred Qualifications:
Candidates with the following skills and qualifications will be preferred:
- Hands-on with Databricks for unified data analytics, including Databricks Notebooks, Delta Lake, and Catalogues
- Proficiency in using the ELK (Elasticsearch, Logstash, Kibana) stack for real-time search, log analysis, and visualization
- Strong background in analytics, including the ability to derive actionable insights from large datasets and support data-driven decision-making
- Experience with data visualization tools like Tableau, Power BI, or Grafana
- Familiarity with Docker for containerization and Kubernetes for orchestration
Estimated Salary: $180,000 - $250,000 per year, depending on location and experience
-
Highly Experienced Data Engineer
9 hours ago
Bengaluru, Karnataka, India Mouri Tech (P) Ltd Full timeCompany Overview:Mouri Tech (P) Ltd is a renowned technology firm seeking an accomplished Senior MSBI Developer to spearhead our data engineering endeavors.Estimated Salary: $160,000 - $200,000 per year, based on industry standards and location.Job Description:We require a highly experienced professional with 9+ years of expertise in SSIS, SAAS, SSRS, and...
-
Highly Experienced Cloud Infrastructure Engineer
3 weeks ago
Bengaluru, Karnataka, India Groww Full timeAbout GrowwWe are a passionate group of people driven by ownership, customer-centricity, integrity and the passion to constantly challenge the status quo. Our long-term vision is to become the trusted financial partner for millions of Indians.Our culture enables us to be what we are — India's fastest-growing financial services company. It fosters an...
-
Highly Experienced Network Engineer
1 week ago
Bengaluru, Karnataka, India Capgemini Full timeAbout the RoleWe are seeking a highly skilled Highly Experienced Network Engineer to join our team at Capgemini. In this role, you will be responsible for designing and implementing robust network infrastructure solutions that meet the needs of our clients.Key ResponsibilitiesYou will have strong knowledge of VLAN, RIP, BGP, OSPF protocols and be able to...
-
Highly Experienced Routing Engineer
4 weeks ago
Bengaluru, Karnataka, India Juniper Networks Full timeJuniper Networks is committed to delivering a new network for the next decade. To achieve this goal, we are seeking a Highly Experienced Routing Engineer to join our team in Bangalore.The ideal candidate will have 8+ years of experience in the Networking industry and a thorough understanding of IP Packet flow, OSI layers, and related technologies. They will...
-
Highly Experienced Software Engineering Lead
3 weeks ago
Bengaluru, Karnataka, India Kredivo Group Full timeAbout the RoleWe are seeking a highly skilled and experienced software engineering lead to join our team at Kredivo Group. As a key member of our organization, you will be responsible for leading large-scale features and projects, collaborating with cross-functional teams, and driving innovation in software development.
-
Highly Experienced AWS Cloud Data Architect
2 days ago
Bengaluru, Karnataka, India Foundit Full timeAbout the Role:We are seeking a highly skilled and experienced AWS Cloud Data Architect to join our dynamic team at Foundit. As a key member of our data engineering team, you will be responsible for designing, implementing, and maintaining large-scale data processing pipelines on AWS.Responsibilities: Develop and implement efficient data ingestion,...
-
Highly Experienced Pega BPM Developer Wanted
2 weeks ago
Bengaluru, Karnataka, India NTT DATA Services Full timeNTT DATA Services is a leading provider of digital and AI infrastructure, committed to helping clients innovate, optimize, and transform for long-term success. As a trusted global innovator of business and technology services, we serve 75% of the Fortune Global 100.We are seeking a highly experienced Pega BPM Developer to join our team in Bangalore, India....
-
Bengaluru, Karnataka, India Covenant Consultants Full timeWe are a leading consulting firm Covenant Consultants, and we are seeking an exceptionally skilled Senior Software Engineer to join our team.Salary: $150,000 - $200,000 per annum.Job Description:About the RoleThis is a rare opportunity for a highly experienced Senior Software Engineer to take on a challenging role that involves building and maintaining...
-
Highly Experienced Software Architect
3 weeks ago
Bengaluru, Karnataka, India Squareroot Consulting Pvt Ltd Full timeJob Title: Highly Experienced Software ArchitectAbout Squareroot Consulting Pvt LtdSquareroot Consulting Pvt Ltd is a reputable consulting firm that offers cutting-edge technology solutions to its clients. We are looking for highly skilled and experienced software architects to join our team.Salary Range: ₹1800000 - ₹2500000 per annumJob...
-
Highly Skilled Azure Data Engineer
4 weeks ago
Bengaluru, Karnataka, India Ascent Force Full timeAbout the RoleWe are seeking an experienced and highly skilled Azure Data Engineer with 5-10 years of expertise in Azure Data Factory (ADF), Databricks, and PySpark to join Ascent Force.
-
Highly Experienced AI Engineer
4 weeks ago
Bengaluru, Karnataka, India Oracle Full timeAt Oracle, we are building a future-ready Gen2 cloud Data Science service platform, and we need your expertise to help us succeed. As a highly skilled Artificial Intelligence (AI) engineer, you will play a crucial role in designing, implementing, and deploying machine learning models in production environments.The ideal candidate will have a strong...
-
Bengaluru, Karnataka, India AECOM Full timeRole OverviewAECOM is seeking a highly skilled and experienced Water Resources Engineer to join our team. As a key member of our engineering team, you will be responsible for contributing to the design and development of large-scale water resource projects.
-
Highly Experienced Software Development Lead
3 weeks ago
Bengaluru, Karnataka, India Synopsys Inc Full timeOverviewSynopsys Inc. is a world-leading electronic design automation (EDA) software company that empowers the global electronics industry's innovation and productivity.Job Title: Highly Experienced Software Development LeadEstimated Salary: ₹45 Lakhs - ₹60 Lakhs per annum,Location: Bangalore, IndiaAbout the Role:We are looking for an exceptional Senior...
-
Bengaluru, Karnataka, India LinkedIn Full timeJob OverviewWe are seeking a highly experienced Senior Software Engineering Manager, Systems Infrastructure to join our team at LinkedIn in Bangalore, India.
-
Bengaluru, Karnataka, India Egis Group Full time**Job Summary**We are seeking a highly experienced structural engineer to join our team at Egis Group. As a key member of our engineering team, you will be responsible for designing and developing structural solutions for complex projects.**Responsibilities:**Design and develop structural solutions for complex projects, ensuring accuracy and adherence to...
-
Bengaluru, Karnataka, India NTT DATA Full timeJob OverviewWe are seeking a highly skilled software engineer to join our team at NTT DATA, specializing in Teamcenter.
-
Highly Skilled Data Engineer
3 weeks ago
Bengaluru, Karnataka, India Thomson Reuters Full timeRole OverviewWe are seeking a highly skilled Data Engineer to join our team at Thomson Reuters.About the RoleThis is an exciting opportunity to work with a talented team of professionals who are passionate about data engineering and analytics.Main ResponsibilitiesDesign, build, and maintain large-scale data pipelines and architectures.Develop and implement...
-
Highly Experienced Full Stack Engineer
1 day ago
Bengaluru, Karnataka, India SixCardinals Full timeAbout SixCardinalsWe are a fast-paced and entrepreneurial startup that values innovation and creativity. Our team is passionate about building cutting-edge technology solutions that transform industries.As a Highly Experienced Full Stack Engineer at SixCardinals, you will lead a team of 4-5 engineers in delivering high-quality solutions end-to-end. This...
-
Highly Experienced Silicon Validation Engineer
4 weeks ago
Bengaluru, Karnataka, India arm limited Full timeAbout ArmArm is a world leader in semiconductor intellectual property (IP) and has more than 1,000 employees worldwide.We have an exciting opportunity for a Highly Experienced Silicon Validation Engineer to join our team!Job DescriptionAs a key member of our team, you will be responsible for driving validation specifications and methodology and delivering on...
-
Highly Skilled Software Developer
3 weeks ago
Bengaluru, Karnataka, India HEVO DATA Full timeAbout Hevo Data:We are a cutting-edge technology company that helps businesses grow by providing fast and intuitive data pipelines. Our mission is to simplify data access and make it easy for everyone to unlock its potential.At Hevo, we deal with massive amounts of data generated by over 2000 customers in 40+ countries. Our platform is used by brands like...