Big Data Engineer – Spark, Kafka
3 days ago
Job description:
Your Experience
5 years of experience in large scale realtime stream processing using Apache Spark with messaging infrastructure like KafkaPulsar
7 years of experience on Data Pipeline development ETL and processing of structured and unstructured data
5 years of experience using NoSQL systems like MongoDB DynamoDB and SnowFlake Athena
Experience with technologies like EMR S3 Lambda AWS Fargate CloudWatch DataDog
Experience owning and implementing technicaldata solutions or pipelines
Excellent written and verbal communication skills in English
Strong work ethic and entrepreneurial spirit
Additional Information Main skills set that are a must are
1 Spark with Scala or Python
2 Apache Kafka
3Snowflake and NoSQL DB experience like MongoDB
4 Experience in AWS cloud services
5 Solid understanding of Medallion architecture Parquet Iceberg
Behavioral Competences
Excellent communication ability to communicate on different levels of seniority
Team player enjoying being part of a crossfunctional setup
Ability to perform well on timecritical endeavors and on multiple fronts at the time
Strong dedication to quality and a client mindset
A passion for learning and continuous improvement
Skills:
Mandatory Skills : Apache Spark,Python,Big Data Hadoop Ecosystem,Data Architecture
-
Big Data Engineer
6 days ago
Bengaluru, Karnataka, India Techno Comp Full time ₹ 20,00,000 - ₹ 25,00,000 per yearJOB RESPONSIBILITY· Experience with Big Data technologies will be a plus (Hadoop, Spark, Kafka, HBase, etc)· Write SQL queries to validate the dashboard output· Working experience with database environment - understanding relational database structure and hands-on SQL knowledge to extract/manipulate data for variance testing.· Performing code reviews and...
-
Big Data Engineer – Spark
3 days ago
Bengaluru, Karnataka, India POWER IT SERVICES Full time ₹ 6,00,000 - ₹ 12,00,000 per yearBig Data Engineer – Spark & AnalyticsLocation- ( Bangalore, Maharashtra)Job Description•Gather operational intel on business processes and policies from multiple sources•Prepare periodical and ad-hoc reports using operational data•Develop semantic core to align data with business processes•Support operations team's work streams for data processing,...
-
Big Data Developer
3 days ago
Bengaluru, Karnataka, India LION AND ELEPHANTS Full time ₹ 20,00,000 - ₹ 25,00,000 per yearJob Description:We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. We have openings in Bengaluru, Chennai, Hyderabad, Pune, Delhi.Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data...
-
Big Data Engineer – Azure
6 days ago
Bengaluru, Karnataka, India Innova ESI Full time ₹ 20,00,000 - ₹ 25,00,000 per yearRole: Big Data Engineer – AzureExperience: 10+ YearsLocation: BangaloreImmediate Joiners OnlyBig Data :Mandatory Skills:Azure Databricks, Azure Data Factory, Azure Function apps, Apache Spark, Scala, Java, Apache Kafka, event stream and Big DataOptional Skills: Airflow, PythonRoles & ResponsibilitiesOverall, experience in IT industry, including 5 years in...
-
Big Data + Azure Engineer
17 hours ago
Bengaluru, Karnataka, India Innova ESI Full time ₹ 12,00,000 - ₹ 36,00,000 per yearRole: Big Data + Azure EngineerLocation: BangaloreExperience: 12+ yearsNotice: ImmediateJob Description:Mandatory Skills:Azure Databricks, Azure Data Factory, Azure Function apps, Apache Spark, Scala, Java, Apache Kafka, event stream and Big DataOptional Skills: Airflow, PythonRoles & ResponsibilitiesOverall, 12 years of experience in IT industry, including...
-
Big Data Engineer(Iceberg)
6 days ago
Bengaluru, Karnataka, India Brillio Full time ₹ 20,00,000 - ₹ 25,00,000 per yearLooking for - Hive, SQL, Python, Spark, Hadoop and kafka with Iceberg experienceBachelor's or Master's degree in Computer Science, Information Systems, or a related field.Design, develop, and maintain data pipelines and ETL processes using Python, Hive, Spark, and other relevant technologies.Good Hands on Exp with javaMinimum of 6+ years of experience as a...
-
Big Data Engineer
3 days ago
Bengaluru, Karnataka, India The IT Firm Full time ₹ 15,00,000 - ₹ 25,00,000 per yearRole OverviewAs a Big Data Engineer, you will be responsible for building and maintaining scalable data pipelines and architectures that support data analytics, reporting, and real-time processing. You will collaborate with data scientists, business analysts, and software engineers to design robust solutions that drive data-driven decisions.Mandatory Skill...
-
Big Data Engineer
2 weeks ago
Bengaluru, Karnataka, India Deqode Full time ₹ 12,00,000 - ₹ 24,00,000 per yearProfile: Big Data Engineer (System Design)Experience:5+ yearsLocation:BangaloreWork Mode:HybridAbout The RoleWe're looking for an experiencedBig Data Engineerwith system design expertise to architect and build scalable data pipelines and optimize big data solutions.Key ResponsibilitiesDesign, develop, and maintain data pipelines and ETL processes using...
-
Kafka Developer
2 days ago
Bengaluru, Karnataka, India Appit Software Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per yearKafka Developer Experienced Associate (36 Years)Our Analytics & Insights Managed Services team brings a unique combination of industry expertise, technology, data management, and managedservices experience to create sustained outcomes for our clients and improve business performance. We empower companies to transform their approach to analytics and insights...
-
Big Data Engineer
7 days ago
Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full timeJob Title :- Data Engineer - PysparkExperience: 5 to 8 YearsLocation: Pune/HyderabadJob DescriptionRequired Skills:5+ years of experience in Big data and pysparkMust-HaveGood work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQLGood-to-HaveGood Spark, Pyspark,Big Data experienceSpark UI/Optimization/debugging techniquesGood...