Lead Data Engineer

4 weeks ago

Kakinada Cochin Kochi Ernakulam, India Prevalent Ai India Full time

Role Purpose and Key Accountabilities:

The primary role of a Lead Data Engineer is to support development of various data engineering components of Prevalent s Security Insight Platform (SIP) across data collection, transport, transformation, ingestion and analytics. SIP is based on open source big data technologies designed to ingest, process and analyse peta byte scale data on a distributed architecture.

The ideal candidate would be a self-motivated individual with strong technology skills, commitment to quality and positive work ethic, who can

  • Participate in design, development, and deployment of SIP data management modules.
  • Develop data ingestion, parsing, scheduling, processing and other data management components based on agile development practices.
  • Work in a fast-paced agile development environment.
  • Conduct unit, system and integration testing of developed modules.
  • Select and integrate any Big Data tools and frameworks required to provide requested capabilities.
  • Collaborate with client and partner teams to ensure high quality data solutions are delivered.
  • Troubleshoot operational escalations and resolve client implementation issues in a timely manner with strong collaboration and care for business priorities.
  • Develop an understanding of industry trends and best practices related to Data Engineering and Open Source Big Data technologies.
  • Create and follow personal education plan in the technology stack and solution architecture.
  • Collaborate and contribute to agile teamwork.
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management.

Skills Experience

  • Significant experience in data engineering on open source big data technologies.
  • Experience with big data tools: Hadoop, Spark, Kafka, etc. and NoSQL databases like Postgres, Mongo DB, Elastic.
  • Experience with data pipeline and workflow management tools: Airflow, etc.
  • Experience with AWS cloud services: EC2, EMR, RDS.
  • Experience with stream-processing systems: Spark-Streaming, etc.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Experience building and optimizing big data data pipelines, architectures and data sets.
  • Great verbal and written communication skills.
  • Self-motivated individual with an analytical mind-set capable of working in a fast-paced environment.
  • Strong project management and organizational skills.

Knowledge:

  • Good knowledge of object-oriented/object function scripting languages: Python, Java, Scala.
  • Good understanding of Data lake concepts such as data ingestion and transformation.
  • Understanding of Hadoop Framework including MapReduce concepts and parallel processing.
  • In depth understanding of Spark Framework.
  • Good understanding of Agile methodology.
  • Working knowledge of message queuing, stream processing, and highly scalable big data data stores.
  • Great verbal and written communication skills.
  • Self-motivated individual with an analytical mind-set capable of working in a fast-paced environment.
  • Strong project management and organizational skills.

Education:

Masters/Bachelor s Degree in Computer Science Engineering

  • Lead Data Engineer

    2 days ago

    Chennai/Cochin/Kochi/Coimbatore/Hyderabad, India DigiMoksha Solutions Full time

    Position : Lead Data Engineer Experience : 10-15 Years Location : Period : Immediate Description : As a Lead Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will lead a team of data engineers to ensure the successful delivery of projects and contribute to the overall data strategy. The...

  • Lead Data Engineer

    1 day ago

    Chennai/Cochin/Kochi/Coimbatore/Hyderabad, IN DigiMoksha Solutions Full time

    Position : Lead Data EngineerExperience : 10-15 YearsLocation : Period : Immediate Description :As a Lead Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will lead a team of data engineers to ensure the successful delivery of projects and contribute to the overall data strategy. The ideal...

  • Data Analyst Fresher

    1 month ago

    ernakulam, India Lead India Private lltd Full time

    Hiring for 17 Data Analyst Fresher / Date Science Engineer/ IT Company Jobs in Coimbatore, Tamil Nadu,Vijayawada, Andhra Pradesh,Tirupati, Andhra Pradesh,Hyderabad, Telangana,Kochi, Kerala,Ernakulam, Kerala,Lucknow, Uttar Pradesh,Puducherry, Pondicherry,Mumbai, Maharashtra,Data Scientist,Data Analyst,fresher,Python Trainer,SQL Data Base Administrator, for...

  • Senior Power BI Data Analyst

    4 weeks ago

    Bangalore/Cochin/Kochi, IN INSIGHT GLOBAL SOLUTIONS Full time

    The On-Account Data Steward plays a key role in supporting the data governance program of one of our customer's accounts. They are responsible for supporting the establishment and maintenance of an effective data governance framework, ensuring data quality, integrity, compliance, data stewardship, and metadata management activities. With a deep...

  • Senior Power BI Data Analyst

    2 weeks ago

    Bangalore/Cochin/Kochi, India INSIGHT GLOBAL SOLUTIONS Full time

    The On-Account Data Steward plays a key role in supporting the data governance program of one of our customer's accounts. They are responsible for supporting the establishment and maintenance of an effective data governance framework, ensuring data quality, integrity, compliance, data stewardship, and metadata management activities. With a deep...

  • .NET Lead Engineer

    4 weeks ago

    Kochi, India Experion Technologies Full time

    Career Opportunities: Lead Data Analyst 1.5+ years Experience as a Data Engineer on the AWS 2. Strong technical expertise in Python and PySpark 3. Experience with Big Data Tools such as Hadoop and Apache Spark (Pyspark) 4. Solid experience with AWS services such as Cloud Formation, S3, Athena, Glue, Glue Data Brew, SQS, SNS, AWS Lake...

  • .NET Lead Engineer

    3 weeks ago

    kochi, India Experion Technologies Full time

    Career Opportunities: Lead Data Analyst 1.5+ years Experience as a Data Engineer on the AWS2. Strong technical expertise in Python and PySpark3. Experience with Big Data Tools such as Hadoop and Apache Spark (Pyspark)4. Solid experience with AWS services such as Cloud Formation, S3, Athena, Glue, Glue Data Brew, SQS, SNS, AWS Lake Formation,...

  • Data Engineer

    1 month ago

    kochi, India Recruiter Wings Consultancy Full time

    Data Engineer: Job Description: 2+ years of hands-on experience with relational, NoSQL, or distributed database systems, preferably BigQuery and PostgreSQL.Strong prociency in SQL, with the ability to write complex queries and optimize database performance.Proven experience with data migration, ETL processes, and data integration. Hands-on experience in...

  • React.js Lead

    2 weeks ago

    Cochin/Kochi, India VacanciesHUB HRM Solutions Pvt Ltd Full time

    Job Title : React JS LeadExperience Required : 6-8 years of overall experience with 4+ years specifically in React JS.Key Responsibilities : Leadership: Lead and mentor a team of React JS developers, providing guidance and support to ensure successful project delivery. Development: Hands-on development experience in React JS and its ecosystem, including...

  • Data Engineer

    2 hours ago

    Kochi, India Electronikmedia (EM) Full time

    Job Description:We are seeking a talented individual with a strong background in Python for ML pipelines, particularly with PyTorch and scikit-learn, as well as expertise in AWS to join our team. The ideal candidate will have experience in ingesting, structuring, and analyzing various unstructured data sources, along with designing and maintaining data...

  • Data Engineering Consultant

    4 days ago

    Kochi, India Brennan IT India Private Limited Full time

    Data Engineering ConsultantPosition: Data Engineering ConsultantWork Location: Navi Mumbai or KochiPURPOSE OF ROLEAs a Data Engineering Consultant, you will deliver projects based on Microsoft’s Azure Data Services. This will include developing components, peer-reviewing code and contributing to the Intellectual Property Library.The function of this role...

  • Data Engineering Consultant

    1 week ago

    kochi, India Brennan IT India Private Limited Full time

    Data Engineering Consultant Position: Data Engineering ConsultantWork Location: Navi Mumbai or KochiPURPOSE OF ROLEAs a Data Engineering Consultant, you will deliver projects based on Microsoft’s Azure Data Services. This will include developing components, peer-reviewing code and contributing to the Intellectual Property Library.The...

  • Data Engineering Consultant

    1 week ago

    Kochi, India Brennan IT India Private Limited Full time

    Data Engineering Consultant Position: Data Engineering Consultant Work Location: Navi Mumbai or Kochi PURPOSE OF ROLE As a Data Engineering Consultant, you will deliver projects based on Microsoft’s Azure Data Services. This will include developing components, peer-reviewing code and contributing to the Intellectual Property Library. The...

  • Senior Php Laravel Developer and Angular developer

    3 weeks ago

    Cochin/ Kochi/ Ernakulam, India timesjobs Full time

    Looking for efficient Laravel developer specialised in developing web applications using Laravel Framework within PHP programming language.Immediate Joiner Preferred.Responsibilities and DutiesDevelop, record and maintain cutting edge web-based PHP applications on portal plus premium service platformsBuild innovative, state-of-the-art applications and...

  • Lead Automation Test Engineer

    4 weeks ago

    Kochi, India Techversant Infotech Pvt Ltd Full time

    Lead Automation Test EngineerAbout Techversant:Techversant is a Global Technology Services and Solutions Company providing high-quality end-to-end IT Solutions and Services to its clients across the globe for more than 15 years.Techversant has been providing Technology services in the key areas of Digital Transformation, Understanding the Evolving Industrial...

  • Lead Generation Internship

    1 week ago

    Cochin, Kerala, India Growcoms Private Limited Full time

    About the internship: Develop and implement lead generation strategies to attract potential customers.Utilize various marketing channels such as email marketing, social media, content marketing, and paid advertising to generate leads.Qualify leads based on predefined criteria and prioritize them for sales engagement.Implement lead scoring methodologies to...

  • Lead Automation Test Engineer

    1 month ago

    Kochi, India Techversant Infotech Pvt Ltd Full time

    Lead Automation Test Engineer About Techversant: Techversant is a Global Technology Services and Solutions Company providing high-quality end-to-end IT Solutions and Services to its clients across the globe for more than 15 years. Techversant has been providing Technology services in the key areas of Digital Transformation, Understanding the...

  • Lead Automation Test Engineer

    1 month ago

    kochi, India Techversant Infotech Pvt Ltd Full time

    Lead Automation Test Engineer About Techversant:Techversant is a Global Technology Services and Solutions Company providing high-quality end-to-end IT Solutions and Services to its clients across the globe for more than 15 years.Techversant has been providing Technology services in the key areas of Digital Transformation, Understanding the...

  • .NET Lead Engineer

    3 weeks ago

    Kochi, India Experion Technologies Full time

    Career Opportunities:Lead Data Analyst1.5+ years Experience as a Data Engineer on the AWS2. Strong technical expertise in Python and PySpark3. Experience with Big Data Tools such as Hadoop and Apache Spark (Pyspark)4. Solid experience with AWS services such as Cloud Formation, S3, Athena, Glue, Glue Data Brew, SQS, SNS, AWS Lake Formation, RDS, Redshift,...

  • Executive - Sales Back End

    3 weeks ago

    Cochin/ Kochi/ Ernakulam, India timesjobs Full time

    Job Profile: Managing the correspondence between the sales team and their clients, processing new leads, monitoring customer accounts, providing data and reports to help the sales team, keeping track of sales targets, answering phone calls.Requirements: Graduates with good communication skills in Hindi, English and Malayalam.Preferred experience: 0 - 2...