Big Data Engineer

4 hours ago


Kozhikode, India HCLTech Full time

We are seeking a talented Scala and Spark Engineer to join our team. The ideal candidate will have a strong background in Scala and Spark development, with experience in building scalable and reliable data processing pipelines. Additionally, familiarity with Google Big Query for backend connectivity is highly desirable.

Responsibilities:

Develop and maintain Scala-based applications and services for data processing and analysis.

Design and implement efficient and scalable Spark jobs for large-scale data processing tasks.

Collaborate with cross-functional teams to integrate data pipelines with Google BigQuery for backend connectivity.

Optimize and tune Spark jobs for performance and efficiency. Troubleshoot and debug issues in production environments.

Stay up-to-date with the latest advancements in Scala, Spark, and BigQuery technologies.

Qualifications:

Bachelor's degree in Computer Science, Engineering, or related field.

Strong proficiency in Scala programming language.

Solid understanding of Apache Spark framework and its ecosystem.

Experience with building and optimising data pipelines using Spark.

Familiarity with Google BigQuery for backend connectivity is a plus.

Knowledge of distributed computing principles and techniques.

Excellent problem-solving and analytical skills.

Strong communication and collaboration skills.

Ability to work effectively in a fast-paced environment and manage multiple priorities.

Preferred Qualifications:

Knowledge of cloud computing platforms such as Google Cloud Platform (GCP)



  • Kozhikode, Kerala, India System Two Security Full time

    Key ResponsibilitiesAs a Senior Security Data Engineer at System Two Security, you will play a pivotal role in ensuring the integrity and usability of our data. You will be responsible for generating synthetic data, building validation frameworks for ML models, partnering with data science teams to tailor AI solutions, and overseeing data architecture and...


  • Kozhikode, India Manuh Technologies Full time

    Role Summary The Senior Data Engineer role is responsible for designing, developing and maintaining data pipelines, databases and data infrastructure to enable the efficient data collection, storage and analysis. This role requires collaboration with data scientists, data infrastructure architects and data consumption stakeholders to ensure the availability...


  • kozhikode, India OneNine AI Full time

    World's leading commercial real estate company is seeking a Senior Data Engineer, with 7+ years of experience, to lead the engineering efforts for a new quantitative research team by building data pipelines, developing ML/AI solutions, and supporting data science models that aid investment decisions in commercial real estate.. The ideal candidate would be an...

  • Data Architect

    2 weeks ago


    Kozhikode, India Job Conquers Full time

    Company Descriptionwe suggest you enter details hereRole DescriptionThis is a full-time remote role for a Data Architect. The Data Architect will be responsible for designing, creating, and managing an organization's data architecture, including databases, data lakes, and data warehouses. They will also collaborate with data scientists, analysts, and IT...

  • AWS Data Engineer

    2 months ago


    Kozhikode, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS Human Resources Team!!Job Role: AWS Data EngineerJob Locations: HyderabadExperience Range: 7 to 10 yearsJob Description:-Key Responsibilities: Lead the design, implementation, and optimization of data pipelines and architectures using AWS services like S3, Redshift, Glue, Lambda, and Step Functions. Architect data solutions...


  • Kozhikode, Kerala, India Aezion, Inc Full time

    Job Title: Advanced Data Engineer LeadAzure Data Engineering Expertise: We seek an experienced Senior Data Engineer Lead to spearhead our data engineering efforts, leveraging expertise in T-SQL, SSIS, SSRS, and Azure technologies.Key Responsibilities:Design, develop, and deploy complex data extraction pipelines in Python.Collaborate with cross-functional...


  • Kozhikode, India PFACTORIAL TECHNOLOGIES PVT LTD Full time

    Job Overview:As a Data Scientist Engineer, you will play a critical role in developing and implementing advanced data science solutions to tackle complex business challenges and drive innovation within our organization. You will collaborate with cross-functional teams, lead machine learning projects, and mentor junior team members while staying at the...


  • Kozhikode, India Mphasis Full time

    Job Summary: We are seeking a highly skilled and experienced Data engineering with Tableau skillsKey Responsibilities: Minimum 10 yrs with Data engineering with Tableau skills Mandatory Skills Primary DevelopsAWS- AWS ( S3, Lambda, Glue, Step function, Redshift, dynamo DB, Quick sight,)/PythonTechnical Skills -CI/CD Pipeline Development:Design, implement,...


  • Kozhikode, Kerala, India interface Full time

    Job DescriptionAt Interface.ai, we provide an Intelligent Virtual Assistant (IVA) to Financial Institutions (FIs) to automate calls and customer inquiries across multiple channels, and engage customers with financial insights and upsell/cross-sell opportunities.Our IVA is transforming financial institutions' call centers from a cost to a revenue center. Our...


  • Kozhikode, India Recro Full time

    Our Client is looking for Proficient in Python, TensorFlow, PyTorch, scikit-learn, Pandas; experienced in AI and ML model development, integration, deployment; familiar with Docker, Kubernetes, CI/CD tools, AWS/Azure/GCP; strong in data structures, algorithms, system design; and knowledgeable in SQL, PostgreSQL, and NoSQL databases.Responsibilities: Model...


  • Kozhikode, Kerala, India Idyllic Services Full time

    At Idyllic Services, we are seeking a highly skilled Data Migration Expert to join our team. This role is ideal for an experienced professional looking to leverage their expertise in data migration and analytics to drive business growth.About the Role:We are looking for a talented individual with strong experience in data migration, particularly in the BFSI...


  • Kozhikode, India Coforge Full time

    Job Title: Network Security EngineerSkills: Network Security, Routing/Switching, Firewalls, SDWANLocation: Greater NoidaExperience: 7-12 YearsNote: Immediate joiners are preferableThe role requires at least 8+ years of hands-on expertise in Network Security, including:Network Design and Development: Strong experience in designing and managing complex...


  • kozhikode, India Coforge Full time

    Job Title: Network Security EngineerSkills: Network Security, Routing/Switching, Firewalls, SDWANLocation: Greater NoidaExperience: 7-12 YearsNote: Immediate joiners are preferableThe role requires at least 8+ years of hands-on expertise in Network Security, including:Network Design and Development: Strong experience in designing and managing complex...


  • Kozhikode, Kerala, India Job Conquers Full time

    Job Conquers is seeking a seasoned Data Architect to spearhead the design, creation, and management of our data architecture.The ideal candidate will possess 15+ years of experience in data architecture and demonstrate expertise in Snowflake, cloud Data Warehousing (AWS), and Data Lakes.Key responsibilities include:Designing and implementing databases, data...

  • DataOps Engineer

    1 day ago


    Kozhikode, India HTC Global Services Full time

    HTC – A brief profileEstablished in 1990, HTC Inc., a CMM Level 5 company with headquarters in Troy, Michigan, is a leading global Information Technology solution and BPO provider. HTC assists clients across multiple industry verticals, offering turnkey project lifecycle in, e-business, data warehousing, embedded systems, ECM, SCM, CRM, and ERP solutions....

  • Pyspark developer

    2 days ago


    Kozhikode, India Tata Consultancy Services Full time

    TCS Greeting!! Desired Competencies (Technical/Behavioral Competency)Must-Have · Sound programming knowledge on PySpark & SQL in terms of processing large amount of semi structured & unstructured data · Ability to design data pipelines in end to end manner · Knowledge on Avro, Parquet format · Knowledge on working on Hadoop Big Data platform · Strong...


  • Kozhikode, Kerala, India ANSR Full time

    OverviewANSR is seeking a highly skilled Network Infrastructure Lead to join our team. As a key member of our technical leadership group, you will be responsible for managing data center and cloud networks, leading projects, and mentoring team members.SalaryWe offer a competitive salary of $120,000 - $180,000 per year, depending on experience and...

  • Software Engineer

    2 weeks ago


    Kozhikode, India Quidich Innovation Labs Full time

    About the CompanyQuidich Innovation Labs is a global company headquartered in Mumbai, India that pioneers in customized technology solutions for Sports Broadcast. From the outset, we have believed in the power of the story that sport tells as a tool to bring people together; and that this story best reaches fans, through the way it is broadcast. Building on...

  • Lead Engineer

    4 weeks ago


    Kozhikode, India Finanshels Accounting Accounting LLP Full time

    Job Description This is a remote position. Company Overview: Finanshels.com is a pioneering financial technology company headquartered in Dubai, UAE. Our mission is to revolutionize the financial landscape by empowering businesses with cutting-edge solutions that streamline accounting and tax operations, democratize access to skilled finance...

  • MIS Analyst Trainee

    2 months ago


    Kozhikode, India XYLEM LEARNING Full time

    Qualification & Eligibility:CGPA is not a disqualificationBranch- Open to all branchesExperience in Excel/GSheet & SQLSkill Sets:Data analytics and Data modelingKnowledge of information systemsBusiness process improvementAdvanced Excel and database managementExperience with data analysis and business automation toolsStrong interpersonal,communication and...