Big Data Developer

2 weeks ago

BangalorePune, IN Xcelyst Full time

Big Data + Scala developer

Experience : 4-8 Yrs. experience overall

Mandatory Skills :


Programming Languages : Scala (SCALA IS MAND)

Big Data Frameworks : Spark, Hadoop/HDFS/Hive/Sqoop/Oozie, Kafka, HBase or any other NoSQL Store

Good to Have : Streaming with Spark/Kafka, Cloud knowledge (AWS/Azure)Scheduling and creatin

Location : Bangalore / Pune

Notice Period : Immediate Joiner - 30 days

Job Description :

We are seeking a highly skilled and motivated Big Data Developer with expertise in Scala to join our dynamic data engineering team. The ideal candidate will have a strong background in developing scalable data processing solutions and a passion for working with large datasets.

Responsibilities :

- Data Processing Pipelines : Design, develop, and maintain data processing pipelines using Scala and Apache Spark.

- Collaboration : Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver effective solutions.

- Documentation : Create and maintain comprehensive documentation for data pipelines, processes, and systems.

- Monitoring and Troubleshooting : Monitor data pipeline performance, troubleshoot issues, and implement solutions to ensure data integrity and system reliability.

- Data Processing : Design and develop data processing pipelines using Scala and big data frameworks like Apache Spark.

- ETL Processes : Implement Extract, Transform, Load (ETL) processes to move and transform data from various sources into a central repository.

- Optimization: Optimize data processing workflows for performance and scalability.

- Data Integration: Integrate diverse data sources, ensuring data quality and consistency.

- Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data needs and deliver solutions workflows with Apache AirFlow

- Excellent communication skills

- Proficiency in Scala programming language.

- Extensive experience with big data technologies such as Apache Spark, Hadoop, Kafka, and HBase.

- Strong understanding of data warehousing concepts and ETL processes.

- Experience with cloud platforms such as AWS, Google Cloud Platform (GCP), or Azure.

- Familiarity with data modeling, data integration, and data quality best practices.

- Analytical Skills : Strong problem-solving and analytical skills, with the ability to troubleshoot and resolve data-related issues.

- Communication Skills : Excellent communication and teamwork skills, with the ability to collaborate effectively with cross-functional teams.

- Attention to Detail : High attention to detail and a commitment to delivering high-quality, reliable data solutions.

- Agile Methodologies : Experience working in an Agile development environment is a plus.

(ref:hirist.tech)

  • Big Data/Databricks Developer

    2 weeks ago

    Bangalore/Pune, IN Xcelyst Full time

    Big Data + Data Bricks DeveloperExperience : 5-8Years- 3+ Years in Big Data Engineer/Min 1 yr in DatabricksNotice Period : Immediate - 30 Days Location : Bangalore / PuneSkills : - Experience in Big Data/Hadoop Ecosystem. Hands on experience in Spark and Hive. Mandatory in DatabricksData Pipeline Development : Design, develop, and maintain scalable and...

  • Big Data Developer

    2 months ago

    Gurgaon/Gurugram/Bangalore, IN Workassist.in Full time

    Our hiring partner is looking for :- This is a full-time on-site role for a Big Data Developer at Big Four Consulting Firm Gurugram.- The Big Data Developer will be responsible for tasks related to Data Engineering, Extract Transform Load (ETL), Big Data, Data Warehousing, and Software Data Engineering, Big Data, Extract Transform Load (ETL), and Data...

  • Big Data Engineer

    7 days ago

    Mumbai/Bangalore/Pune, IN Techno Wise Full time

    Job Role : Big Data EngineerRelevant Experience : 3+ yearsLocation : Navi Mumbai /Bengaluru/PuneNotice Period : Immediate or serving Notice PeriodPrimary Skills : Big Data, PySpark, Cloud- Azure or AWS, Databricks, SQL, PythonOverview : We are seeking a talented Big Data Engineer proficient in PySpark to join our dynamic team. The ideal candidate will play a...

  • Senior Big Data Engineer

    3 weeks ago

    Bangalore/Pune, IN Recrud Full time

    Senior Big Data Engineer Position. Experience Required: 6 -10 Years. Employment Type: Full-Time. About Our Client:. Our client is a distinguished product engineering consulting firm based in Bangalore, dedicated to providing innovative offshore development solutions. With over half a century of combined expertise in product engineering and consulting...

  • Senior Big Data Engineer

    2 months ago

    Bangalore/Pune, IN Recrud Full time

    Senior Big Data Engineer Position. Experience Required: 6 -10 Years. Employment Type: Full-Time. About Our Client:. Our client is a distinguished product engineering consulting firm based in Bangalore, dedicated to providing innovative offshore development solutions. With over half a century of combined expertise in product engineering and consulting...

  • Data Architect

    3 weeks ago

    Bangalore/Hyderabad/Pune, IN Blanket Technologies Full time

    Responsibilities :- Lead a team of data engineers in designing and implementing scalable data solutions. - Collaborate with data scientists to understand data requirements and ensure data integrity. - Develop and maintain data pipelines for efficient data processing. - Data Quality and Governance - Handling customer sensitive ( PII ,GDPR,HIPAA ) with...

  • Data Architect

    3 weeks ago

    Bangalore/Hyderabad/Pune, IN Blanket Technologies Full time

    Responsibilities :- Lead a team of data engineers in designing and implementing scalable data solutions. - Collaborate with data scientists to understand data requirements and ensure data integrity. - Develop and maintain data pipelines for efficient data processing. - Data Quality and Governance - Handling customer sensitive ( PII ,GDPR,HIPAA ) with...

  • Big Data Engineer

    7 days ago

    Mumbai/Pune/Bangalore, IN Techno Wise Full time

    Job Description : We are seeking a talented Big Data Engineer proficient in PySpark to join our dynamic team. The ideal candidate will play a pivotal role in designing, implementing, and maintaining scalable data solutions leveraging the Big Data technologies like PySpark. This role requires a strong understanding of big data technologies, data engineering...

  • Big Data Engineer

    2 months ago

    Bangalore/Chennai/Pune, IN ADVANSOFT Full time

    We are hiring for Data engineer with top most MNC client;Exp : 5+yearsNp : immediate to 15daysLocation : & Responsibilities : - 5+ years of working experience in ETL scalable data pipeline usin61Scala, Python, Pyspark, Hadoop, Apache Spark, Spark SQL, Kafka, Nill, and incremental Data Load with Big data technologies.- Experience working with Databases like...

  • Big Data Engineer

    3 weeks ago

    Bangalore/Chennai/Pune, IN ADVANSOFT Full time

    We are hiring for Data engineer with top most MNC client;Exp : 5+yearsNp : immediate to 15daysLocation : & Responsibilities : - 5+ years of working experience in ETL scalable data pipeline usin61Scala, Python, Pyspark, Hadoop, Apache Spark, Spark SQL, Kafka, Nill, and incremental Data Load with Big data technologies.- Experience working with Databases like...

  • Big Data Engineer

    3 days ago

    Bangalore/Pune, IN Confidentail Full time

    Experience : 4+ YearsLocations : Pune & BangaloreWorking Mode : Hybrid Job Responsibilities :- Design, develop, and maintain highly scalable data processing applications- Write efficient, reusable and well-documented code- Deliver big data projects using Spark, Scala, Python, SQL, HQL, Hive- Maintain and tune existing Hadoop/Hbase applications- Work closely...

  • Big Data Architect

    2 months ago

    Bangalore/Anywhere in India/Multiple Locations, IN ARR Recruitment Solutions Full time

    About the Role :- We are seeking a highly experienced Big Data Architect to join our team and lead the design, development, and implementation of our big data solutions.- This role is ideal for a candidate with a deep understanding of big data technologies and a proven track record of delivering scalable and robust big data architectures.Responsibilities :-...

  • Big Data Architect

    3 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN ARR Recruitment Solutions Full time

    About the Role :- We are seeking a highly experienced Big Data Architect to join our team and lead the design, development, and implementation of our big data solutions.- This role is ideal for a candidate with a deep understanding of big data technologies and a proven track record of delivering scalable and robust big data architectures.Responsibilities :-...

  • Big Data Quality Assurance Engineer

    2 months ago

    Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Big Data Quality Assurance Engineer

    3 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Lead Data Engineer

    1 month ago

    Chennai/Bangalore/Hyderabad, IN Cynosure Corporate Solutions Full time

    Job Description :- 10+ years of design & development experience with big data technologies like Azure, AWS or GCP- Preferred is Azure & Databricks, with experience in Azure DevOps- Experience in data visualizing technology in DL, like PowerBI- Proficient in Python, PySpark and SQL- Proficient in querying and manipulating data from various DB (relational and...

  • Lead Data Engineer

    3 weeks ago

    Chennai/Bangalore/Hyderabad, IN Cynosure Corporate Solutions Full time

    Job Description :- 10+ years of design & development experience with big data technologies like Azure, AWS or GCP- Preferred is Azure & Databricks, with experience in Azure DevOps- Experience in data visualizing technology in DL, like PowerBI- Proficient in Python, PySpark and SQL- Proficient in querying and manipulating data from various DB (relational and...

  • Impetus - Big Data Developer - SQL/Spark

    4 weeks ago

    Bangalore/Gurgaon/Gurugram, IN Impetus Technologies India Pvt. Ltd Full time

    Job Description : - Experience with design and coding across one or more platforms and languages (e.g. Java, Spark/SQL) as appropriate- Hands-on expertise with application design, software development and automated testing Proficient in Big Data technologies - Designs, codes, tests, corrects and documents large and/or complex programs and program...

  • Big Data Engineer

    3 weeks ago

    Any Location/Bangalore, IN Careerfit.ai Full time

    Responsibilities :Solution Design :- Design, develop, and maintain scalable big data solutions and data pipelines for efficient data ingestion, processing, storage, and analysis.- Collaborate with data scientists, analysts, and business stakeholders to identify data requirements, use cases, and business objectives.- Translate business needs into technical...

  • Big Data Engineer

    2 months ago

    Any Location/Bangalore, IN Careerfit.ai Full time

    Responsibilities :Solution Design :- Design, develop, and maintain scalable big data solutions and data pipelines for efficient data ingestion, processing, storage, and analysis.- Collaborate with data scientists, analysts, and business stakeholders to identify data requirements, use cases, and business objectives.- Translate business needs into technical...