Data Software Engineer

1 week ago

Chennai, Tamil Nadu, India One Click AI Full time

Job Title :

Data Software Engineer Spark, Python, (AWS, Kafka / Azure Databricks / GCP)

Job Description :

We are seeking a highly skilled and experienced Data Software Engineer with a strong background in Big Data technologies, distributed systems, and cloud platforms. The ideal candidate will possess excellent programming skills, a solid understanding of data pipelines, and the ability to lead complex projects.

Key Responsibilities :

- Develop, optimize, and maintain big data pipelines using Apache Spark and Python.

- Implement distributed computing solutions and large-scale data processing.

- Build and maintain stream-processing systems using Spark Streaming or Apache Storm.

- Integrate data from diverse sources including RDBMS (SQL Server, Oracle), ERP systems, and file-based sources.

- Work with messaging systems like Kafka or RabbitMQ.

- Develop and optimize SQL queries, stored procedures, and database schemas.

- Utilize NoSQL databases such as HBase, Cassandra, and MongoDB.

- Perform performance tuning of Spark jobs and queries.

- Use cloud-native services (AWS, GCP, or Azure Databricks) for scalable data solutions.

- Design and implement end-to-end Big Data solutions in agile environments.

- Provide technical leadership and mentor junior engineers.

Required Skills & Experience:

- 512 years of experience in Big Data & related technologies

- Expert-level proficiency in Apache Spark (minimum 5 years)

- Hands-on Python programming experience (5 years)

- Cloud experience with at least one of AWS, Azure Databricks, or GCP (3+ years)

- Experience with Apache Kafka or similar messaging platforms (3+ years)

- Familiarity with Hadoop v2, HDFS, MapReduce, Sqoop

- Experience in ETL frameworks and data ingestion pipelines

- Proficient in SQL, data modeling, and working with structured/unstructured data

- Good knowledge of Hive, Impala, and other big data querying tools

- Exposure to Agile methodologies

Preferred Qualifications :

- Strong analytical and problem-solving skills

- Ability to lead teams and manage stakeholders

- Certifications in cloud platforms (AWS, Azure, or GCP) are a plus

Job Categories :

- Data Engineer

- ML/AI Engineer

- Software Engineer

- Data Scientist

Tech Stack Expertise :

Technology Experience Required

- Python 5+ Years

- Apache Spark 5+ Years

- AWS / Azure / GCP 3+ Years

- PySpark 3+ Years

- Apache Kafka 3+ Years

- NoSQL (HBase, MongoDB, Cassandra) Relevant Experience

(ref:hirist.tech)

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    HiRole - Azure Data engineerLocation - Chennai, Gurugram (5days Onsite)Full timeImmediate joiner Only (15days Notice period only)Technical Skills:o Proficiency in SQL for complex queries, database design, and optimization.o Strong experience with PySpark for data transformation and processing.o Hands-on experience with Databricks for building and managing...

  • Senior Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Full time US$ 1,50,000 - US$ 2,00,000 per year

    About The Role:Eucloid is looking for a senior Data Engineer with hands-on expertise in Databricks to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream...

  • Data Engineer

    5 days ago

    Chennai, Tamil Nadu, India NTT DATA North America Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN)."Job Duties: Key Responsibilities:•  ...

  • Software Engineer Data

    2 weeks ago

    Chennai, Tamil Nadu, India R Systems Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Your Role and ImpactYour ContributionAbout The Role–-As a Software Engineer II - Data, you will contribute to the design and development of data systems including pipelines, APIs, analytics, AI and machine learning at scale. You'll be a core part of our Data Products team, building and maintaining production-grade pipelines and platform components that...

  • Senior Data Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India APPIT Software Inc Full time

    Senior Data EngineerLocation: ChennaiJob Type: Contract, 5 monthsExperience: 5-7 yearsJob Summary:We are hiring a Senior Data Engineer to architect enterprise data integration initiatives. This role requires deep technical expertise in data engineering. Familiarity with Oracle Data Integrator (ODI) is preferred, especially in environments using the Oracle...

  • Senior Data Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India APPIT Software Inc Full time

    Senior Data Engineer Location: Chennai Job Type: Contract, 5 months Experience: 5-7 years Job Summary: We are hiring a Senior Data Engineer to architect enterprise data integration initiatives. This role requires deep technical expertise in data engineering. Familiarity with Oracle Data Integrator (ODI) is preferred, especially in environments using...

  • Senior Software Engineer, Product Development

    13 hours ago

    Chennai, Tamil Nadu, India Smart Data Solutions LLC Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Senior Software Engineer, Product DevelopmentSmart Data Solutions, a leading provider of data management, claim routing and workflow solutions to health plans and TPAs, is looking for a Senior Software Engineer, Product Development.What you'll be doing?Should be capable of independently coding and testing in Java as an individual contributor.Lead a team of...

  • Software Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Webasto Roof Systems Full time ₹ 274 - ₹ 1,234 per year

    We are looking for a skilled software engineer who, along with our excellent software development team, will be responsible for working on projects that are currently being developed on by our company. Duties will include but are not limited to developing and directing software system validation and testing methods, as well as directing our software...

  • Software Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Global Software Solutions Group Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    We're looking for a talented Software Engineerto join our team in Chennai and contribute to innovative projects for a leading banking client. If you're passionate about building scalable, high-performance web applications and enjoy working in dynamic environments, this role is for youRequirementsWhat we're looking for: Bachelor's or Master's degree in...

  • GCP Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India CustomerLabs 1P Data OPs Full time

    Position Overview:"Yesterday is history, tomorrow is a mystery, but today is a gift. That's why we call itthe present." - Master OogwayJoin CustomerLabs' dynamic data team as a Data Engineer and play a pivotal role intransforming raw marketing data into actionable insights that power our digitalmarketing platform. As a key member of our data infrastructure...