Software Development Engineer II

1 day ago

Tiruchirappalli, India ClearDemand Full time

Job Summary:Data is the foundation of our business, and your work will ensure that we continue to deliver high-quality competitive intelligence at scale. Web platforms are constantly evolving, deploying sophisticated anti-bot measures—your job is to stay ahead of them. If you thrive on solving complex technical challenges and enjoy working with real-world data at an immense scale, this role is for you.We seek a Software Development Engineer with expertise in cloud infrastructure, Big Data and web crawling technologies. This role bridges system reliability engineering with scalable data extraction solutions, ensuring our infrastructure remains robust and capable of handling high-volume data collection. You will design resilient systems, optimize automation pipelines, and tackle challenges posed by advanced bot-detection mechanisms.Key Responsibilities:Architect, deploy, and manage scalable cloud environments (AWS/GCP/DO) to support distributed data processing solutions to handle terabyte-scale datasets and billions of records efficientlyAutomate infrastructure provisioning, monitoring, and disaster recovery using tools like Terraform, Kubernetes, and Prometheus.Optimize CI/CD pipelines to ensure seamless deployment of web scraping workflows and infrastructure updates.Develop and maintain stealthy web scrapers using Puppeteer, Playwright, and headless chromium browsers.Reverse-engineer bot-detection mechanisms (e.g., TLS fingerprinting, CAPTCHA solving) and implement evasion strategies.Monitor system health, troubleshoot bottlenecks, and ensure 99.99% uptime for data collection and processing pipelines.Implement security best practices for cloud infrastructure, including intrusion detection, data encryption, and compliance audits.Partner with data collection, ML and SaaS teams to align infrastructure scalability with evolving data needsQualifications & Experience:4–6 years of experience in site reliability engineering and cloud infrastructure management Proficiency in Python, JavaScript for scripting and automation . Hands-on experience with Puppeteer/Playwright, headless browsers, and anti-bot evasion techniques . Knowledge of networking protocols, TLS fingerprinting, and CAPTCHA-solving frameworks . Experience with monitoring and observability tools such as Grafana, Prometheus, Elasticsearch, and familiarity with monitoring and optimizing resource utilization in distributed systems.Experience with data lake architectures and optimizing storage using formats such as Parquet, Avro, or ORC.Strong proficiency in cloud platforms (AWS, GCP, or Azure) and containerization/orchestration (Docker, Kubernetes).Deep understanding of infrastructure-as-code tools (Terraform, Ansible) . Deep experience in designing resilient data systems with a focus on fault tolerance, data replication, and disaster recovery strategies in distributed environments.Experience implementing observability frameworks, distributed tracing, and real-time monitoring tools.Excellent problem-solving abilities, with a collaborative mindset and strong communication skills.

  • Software Engineer

    2 weeks ago

    Tiruchirappalli, India Discoveries Quintessential Full time

    Software Engineer (Linux/Unix & Bash Focused) 💻🐧 - Join Our AI Team!Position :Software Engineer (Linux/Unix & Bash Focused) 🛠️ Location-Mumbai, India 🇮🇳 (Preferably Mumbai-based candidates)Job Type-Contractual (1-month probation with potential for Full-Time Conversion! 🚀)Experience -Minimum 1+ years of professional software engineering...

  • Software Engineer

    2 weeks ago

    Tiruchirappalli, India Discoveries Quintessential Full time

    Software Engineer (Linux/Unix & Bash Focused) 💻🐧 - Join Our AI Team!Position :Software Engineer (Linux/Unix & Bash Focused) 🛠️ Location-Mumbai, India 🇮🇳 (Preferably Mumbai-based candidates)Job Type-Contractual (1-month probation with potential for Full-Time Conversion! 🚀)Experience -Minimum 1+ years of professional software engineering...

  • Senior Software Engineer-AWS Backend

    3 weeks ago

    Tiruchirappalli, India IntraEdge Full time

    Job Title: Senior Software Engineer-AWS Backend📍 Location: Pune (Hybrid) 🧑 💻 Experience: 5–8 Years 📅 Employment Type: Full-timeSenior Software EngineerWe are seeking an experienced and motivated Senior Software Engineer with strong expertise in backend engineering, cloud-native development, and SaaS application design. The ideal candidate will...

  • Lead Software Engineer

    1 week ago

    Tiruchirappalli, India DDN Full time

    This is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a global market leader renowned for powering many of the world's most demanding AI data centers, in industries ranging from life sciences and healthcare to financial...

  • ▷ Urgent Search! Senior SQL Server Analysis Services

    2 weeks ago

    Tiruchirappalli, India Sonata Software Full time

    Greetings!!! Currently we have an Urgent Opening for Senior SQL Server Analysis Services (SSAS) Role Developer role with one of our projects, location based out of Hyderabad. Kindly find below the Details for your Perusal. Sonata Software Overview In today's market, there is a unique duality in technology adoption. On one side, extreme focus on cost...

  • Java Software Engineer

    3 weeks ago

    Tiruchirappalli, India Recro Full time

    About the Role: -Understand complex business requirements, partnering with global stakeholders and product team, and offering solutions, all the way from design to implementation, and rollout-Code, test and implement high-quality Java modules for a complex enterprise application system, that is integrated to multiple 3rd party webservices and also supports a...

  • Java Software Engineer

    3 weeks ago

    Tiruchirappalli, India Recro Full time

    About the Role: -Understand complex business requirements, partnering with global stakeholders and product team, and offering solutions, all the way from design to implementation, and rollout-Code, test and implement high-quality Java modules for a complex enterprise application system, that is integrated to multiple 3rd party webservices and also supports a...

  • Urgent Search: Software Development Lead

    3 weeks ago

    Tiruchirappalli, India SAEL Full time

    About the Role We are seeking an experienced Lead – Software Development with 8–10 years of proven expertise in designing and delivering scalable, secure, and high-performing web applications. In this role, you will lead a team of developers, define technical direction, and ensure best practices while staying hands-on in both front-end and back-end...

  • Java Software Engineer

    3 weeks ago

    Tiruchirappalli, India Tata Consultancy Services Full time

    Greetings, TCS is looking for Java API Management & Microservices DeveloperExperience: 5 - 8 years OnlyEducation: Minimum 15 years of full time education (10th, 12th and Graduation)Location: Chennai/Bangalore/Hyderabad/Delhi/PuneMust-Have:Excellent grasp on Core Java8 & hands-on experience developing standalone & web applications, microservices.Hands-on...

  • Full Stack Engineer

    1 week ago

    Tiruchirappalli, India Infogain Full time

    Job Profile Summary: (7-10 Years Only) · In this role, you will design, develop and provide support for Point of Sales platform/s for client to power Sales Process done by life Insurance Distribution channels. This is a hands-on software engineering role. · We are looking for an Engineer who is passionate around solving business problems through innovation...