Lead Big Data/Data Engineer

3 days ago

RemoteBangalorePune, IN TechTiv Software Solutions Full time

The Opportunity :

Digit88 development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA).


The candidate would be joining an existing team of outstanding data engineers in the US and help us expand the data engineering team and work on different products and on different layers of the infrastructure.

Job Profile :

Digit88 is looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for us to deal with the huge structured data sets that we use as an input to accurately generate analytics at scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them.


You will also be responsible for integrating them with the architecture used across the company. Applicants must have a passion for engineering with accuracy and efficiency, be highly motivated and organized, able to work as part of a team, and also possess the ability to work independently with minimal supervision.

To be successful in this role, you should possess :

- Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.

- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale.

- Design and implement systems to cleanse, process, and analyze large data sets using distributed processing tools like Akka and Spark.

- Understanding and critically reviewing existing data pipelines, and coming up with ideas in
collaboration with Technical Leaders and Architects to improve upon current bottlenecks

- Take initiatives, and show the drive to pick up new stuff proactively, and work as a Senior Individual contributor on the multiple products and features we have.

- 7+ years of experience in developing highly scalable Big Data pipelines.

- In-depth understanding of the Big Data ecosystem including processing frameworks like Spark, Akka, Storm, and Hadoop, and the file types they deal with.

- Experience with ETL and Data pipeline tools like Apache NiFi, Airflow etc.

- Excellent coding skills in Java or Scala, including the understanding to apply appropriate Design Patterns when required.

- Experience with Git and build tools like Gradle/Maven/SBT.

- Strong understanding of object-oriented design, data structures, algorithms, profiling, and optimization.

- Have elegant, readable, maintainable and extensible code style

(ref:hirist.tech)

  • Big Data Engineer

    2 weeks ago

    Mumbai/Bangalore/Pune, IN Techno Wise Full time

    Job Role : Big Data EngineerRelevant Experience : 3+ yearsLocation : Navi Mumbai /Bengaluru/PuneNotice Period : Immediate or serving Notice PeriodPrimary Skills : Big Data, PySpark, Cloud- Azure or AWS, Databricks, SQL, PythonOverview : We are seeking a talented Big Data Engineer proficient in PySpark to join our dynamic team. The ideal candidate will play a...

  • Data Architect

    4 weeks ago

    Bangalore/Hyderabad/Pune, IN Blanket Technologies Full time

    Responsibilities :- Lead a team of data engineers in designing and implementing scalable data solutions. - Collaborate with data scientists to understand data requirements and ensure data integrity. - Develop and maintain data pipelines for efficient data processing. - Data Quality and Governance - Handling customer sensitive ( PII ,GDPR,HIPAA ) with...

  • Data Architect

    4 weeks ago

    Bangalore/Hyderabad/Pune, IN Blanket Technologies Full time

    Responsibilities :- Lead a team of data engineers in designing and implementing scalable data solutions. - Collaborate with data scientists to understand data requirements and ensure data integrity. - Develop and maintain data pipelines for efficient data processing. - Data Quality and Governance - Handling customer sensitive ( PII ,GDPR,HIPAA ) with...

  • Java Developer

    3 days ago

    Remote, IN Gama Advisory Services LLP Full time

    Overview :We are looking for a talented Java Developer with 2-3 years of experience in Big Data technologies to join our team. As a Java Developer specializing in Big Data, you will be responsible for designing, developing, and maintaining scalable applications and data pipelines using Java and Big Data frameworks. Your expertise in Java programming and Big...

  • Big Data/ETL Test Engineer

    3 days ago

    Remote, IN Gama Advisory Services LLP Full time

    Overview :We are seeking a proficient Big Data/ETL Test Engineer with 4-6 years of experience in testing Big Data platforms and ETL processes, specifically using Hadoop and Hive technologies. As a Test Engineer, you will play a critical role in ensuring the quality and reliability of our Big Data solutions by designing and executing comprehensive test...

  • Aptus Data Labs

    3 days ago

    Remote, IN Aptus Data Labs Full time

    Description :We are seeking a highly skilled and experienced individual to join our team as a Data Governance expert with AWS knowledge. In this role, you will play a critical role in ensuring data integrity, quality, and security across our organization's data assets. As a data governance expert, your main responsibility will be to develop and implement...

  • Senior Big Data Engineer

    1 day ago

    Remote, IN MindBrain Full time

    Job Title - Big Data Engineer. Location - Remote. Contractual - Duration 1 yr. Responsibilities :- Design, build, and maintain large-scale data processing systems (batch and streaming) using technologies such as Hadoop, Spark, Kafka, etc.- Develop and optimize data pipelines, data ingestion processes, and data integration workflows.- Implement data...

  • Senior Big Data Engineer

    1 day ago

    Mumbai/Bangalore/Pune, IN Techno Wise Full time

    Job Role : Big Data EngineerPosition : Big Data EngineerRelevant Experience : 5+ yearsLocation : Navi Mumbai /Bengaluru/PuneNotice Period : Immediate or serving Notice PeriodPrimary Skills : Big Data, PySpark, Cloud- Azure or AWS, Databricks, SQL, PythonOverview : We are seeking a talented Big Data Engineer proficient in PySpark to join our dynamic team. The...

  • Big Data Engineer

    2 weeks ago

    Mumbai/Pune/Bangalore, IN Techno Wise Full time

    Job Description : We are seeking a talented Big Data Engineer proficient in PySpark to join our dynamic team. The ideal candidate will play a pivotal role in designing, implementing, and maintaining scalable data solutions leveraging the Big Data technologies like PySpark. This role requires a strong understanding of big data technologies, data engineering...

  • Senior Big Data Engineer

    4 weeks ago

    Bangalore/Pune, IN Recrud Full time

    Senior Big Data Engineer Position. Experience Required: 6 -10 Years. Employment Type: Full-Time. About Our Client:. Our client is a distinguished product engineering consulting firm based in Bangalore, dedicated to providing innovative offshore development solutions. With over half a century of combined expertise in product engineering and consulting...

  • Senior Big Data Engineer

    2 months ago

    Bangalore/Pune, IN Recrud Full time

    Senior Big Data Engineer Position. Experience Required: 6 -10 Years. Employment Type: Full-Time. About Our Client:. Our client is a distinguished product engineering consulting firm based in Bangalore, dedicated to providing innovative offshore development solutions. With over half a century of combined expertise in product engineering and consulting...

  • Publicis Sapient

    4 weeks ago

    Bangalore/Gurgaon/Gurugram, IN Publicis Sapient Full time

    Job Description :As Manager, Data Engineering, you will be responsible for translating client requirements into design, architecting, and implementing Azure Cloud based big data solutions for clients. Your role will be focused on delivering high quality solutions by independently driving design discussions related to Data Ingestion, Transformation &...

  • Big Data Developer

    3 weeks ago

    Bangalore/Pune, IN Xcelyst Full time

    Big Data + Scala developer Experience : 4-8 Yrs. experience overallMandatory Skills :Programming Languages : Scala (SCALA IS MAND)Big Data Frameworks : Spark, Hadoop/HDFS/Hive/Sqoop/Oozie, Kafka, HBase or any other NoSQL StoreGood to Have : Streaming with Spark/Kafka, Cloud knowledge (AWS/Azure)Scheduling and creatinLocation : Bangalore / PuneNotice Period :...

  • Lead Data Engineer

    2 months ago

    Chennai/Bangalore/Hyderabad, IN Cynosure Corporate Solutions Full time

    Job Description :- 10+ years of design & development experience with big data technologies like Azure, AWS or GCP- Preferred is Azure & Databricks, with experience in Azure DevOps- Experience in data visualizing technology in DL, like PowerBI- Proficient in Python, PySpark and SQL- Proficient in querying and manipulating data from various DB (relational and...

  • Lead Data Engineer

    4 weeks ago

    Chennai/Bangalore/Hyderabad, IN Cynosure Corporate Solutions Full time

    Job Description :- 10+ years of design & development experience with big data technologies like Azure, AWS or GCP- Preferred is Azure & Databricks, with experience in Azure DevOps- Experience in data visualizing technology in DL, like PowerBI- Proficient in Python, PySpark and SQL- Proficient in querying and manipulating data from various DB (relational and...

  • Data Scientist

    3 days ago

    Remote, IN Georgia IT, Inc. Full time

    Job Description :- We are seeking a skilled Data Scientist / Data Analyst with a solid background in Python programming, SQL, and data visualization. - The ideal candidate will have at least 3 years of relevant experience in data science or data analysis, with specific expertise in NLP techniques and LLM. - This role requires a creative problem-solver who...

  • Big Data Engineer

    2 months ago

    Bangalore/Chennai/Pune, IN ADVANSOFT Full time

    We are hiring for Data engineer with top most MNC client;Exp : 5+yearsNp : immediate to 15daysLocation : & Responsibilities : - 5+ years of working experience in ETL scalable data pipeline usin61Scala, Python, Pyspark, Hadoop, Apache Spark, Spark SQL, Kafka, Nill, and incremental Data Load with Big data technologies.- Experience working with Databases like...

  • Big Data Engineer

    4 weeks ago

    Bangalore/Chennai/Pune, IN ADVANSOFT Full time

    We are hiring for Data engineer with top most MNC client;Exp : 5+yearsNp : immediate to 15daysLocation : & Responsibilities : - 5+ years of working experience in ETL scalable data pipeline usin61Scala, Python, Pyspark, Hadoop, Apache Spark, Spark SQL, Kafka, Nill, and incremental Data Load with Big data technologies.- Experience working with Databases like...

  • Big Data Quality Assurance Engineer

    2 months ago

    Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Big Data Quality Assurance Engineer

    4 weeks ago

    Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...