Manager-Data Engineer

1 week ago


Kolkata, West Bengal, India Genpact Full time

With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world's biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We're harnessing the power of technology and humanity to create meaningful transformation that moves us forward in our pursuit of a world that works better for people. Now, we're calling upon the thinkers and doers, those with a natural curiosity and a hunger to keep learning, keep growing., People who thrive on fearlessly experimenting, seizing opportunities, and pushing boundaries to turn our vision into reality. And as you help us create a better world, we will help you build your own intellectual firepower.

Inviting applications for the role of Manager-Data Engineer

In this role, you will be responsible for extracting information from various websites.

Responsibilities

·Responsible for extracting and ingesting data from websites using web crawling tools.

·In this role you will own the creation process of these tools, services, and workflows to improve crawl/ scrape analysis, reports and data management.

·Responsible for data testing and the scrape to ensure accuracy and quality.

·Developing process to identify and rectify any issues with breaks as well as scale scrapes as needed.

·Develop automated tool to repeat the web crawling/scraping process.

Qualifications

Minimum qualifications

·B.Tech/B.E degree or B.Sc in Computer Science.

Preferred qualifications

§Experience running large scale web scrapes.

§Solid Python knowledge

§Familiarity HTML, CSS, JavaScript.

§Familiarity with techniques and tools for crawling, extracting, and processing data (e.g., Scrapy, pandas, SQL, BeautifulSoup etc).

§Experience with version control, open-source practices, and code review

§Experience with applications designed to display archived web content.

§Great communication skills (written and Spoken in English).

§Bachelor's degree in computer science or a related field or the equivalent demonstrated experience.



  • Kolkata, West Bengal, India NTT DATA Full time

    Job Description NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our...


  • Kolkata, West Bengal, India The Search Manager Full time

    Play a pivotal execution role in setting up & commissioning of Process Engineering Lab, training of manpower to achieve the results right first time. Understand the requirement of Internal and External stakeholders and develop the process and procedures from Standardization & technical cell purview as per the established Integrated management system...

  • Data Engineer

    1 week ago


    Kolkata, West Bengal, India CA-One India Full time

    Job Title : Data : Kolkata, India Responsibilities : Design, build, and maintain scalable data pipelines. Develop SparkSQL queries and BigQuery ML models. Integrate Kafka for realtime data processing. Manage BigQuery Data Transfer Service and Snowflake SQL queries. Collaborate with crossfunctional teams on data : Bachelor's degree in Computer Science or...

  • Data Engineer

    1 week ago


    Kolkata, West Bengal, India Webologix Ltd INC Full time

    Job Position: Data engineer with AWS CloudJob Type: Full-TimeLocation: KolkataExperience: - 5+ to 10+ YearsMust Have: Aws-DataServices, AWS Cloud, Data engineer.Key ResponsibilitiesAWS AuroraAWS RDSAWS Dynamo DBAWS S3AWS LambdaAWS GlueAWS Step FunctionsAWS Event BridgeAWS DMSAWS SCTData MigrationPythonPySparkSQLEducation:B. Tech or equivalent IT Degree.

  • Data Engineer

    1 week ago


    Kolkata, West Bengal, India Webologix Ltd INC Full time

    Job Position: Data engineer with AWS CloudJob Type: Full-TimeLocation: KolkataExperience: - 5+ to 10+ YearsMust Have: Aws-Data-Services, AWS Cloud, Data engineer.Key ResponsibilitiesAWS AuroraAWS RDSAWS Dynamo DBAWS S3AWS LambdaAWS GlueAWS Step FunctionsAWS Event BridgeAWS DMSAWS SCTData MigrationPythonPySparkSQLEducation:B. Tech or equivalent IT Degree.

  • Data Engineer

    1 week ago


    Kolkata, West Bengal, India Webologix Ltd INC Full time

    Job Position: Data engineer with AWS Cloud Job Type: Full-Time Location:Kolkata Experience:5+ to 10+ Years Must Have: Aws-DataServices, AWS Cloud, Data engineer.Key Responsibilities AWS Aurora AWS RDS AWS Dynamo DB AWS S3 AWS Lambda AWS Glue AWS Step Functions AWS Event Bridge AWS DMS AWS SCT Data Migration Python PySpark SQL Education: B. Tech or equivalent...

  • Data Engineer

    1 week ago


    Kolkata, West Bengal, India Mejoba Consulting Full time

    Job description :We seek a talented Data Engineer with expertise in Robotic Process Automation (RPA) and Artificial Intelligence/Machine Learning (AI/ML) to join our dynamic team.As a Data Engineer with RPA and AI/ML expertise, you will play a crucial role in designing, implementing, and maintaining data pipelines, workflows, and automation solutions.You...

  • GCP Data Engineer

    1 week ago


    Kolkata, West Bengal, India LTIMindtree Full time

    Job Title: GCP SrData Engineer Experience:5-12 years Education: Bachelor of engineering in any discipline/equivalent Location: Bangalore, Chennai, Mumbai, Pune, Kolkata, Hyderabad, Delhi Desired Candidate Profile Job Description: Overall, more than 5+ Yrs of experience in Data projects. Good Knowledge of GCP Bigquery, SQL, Python, dataflow skills. Has worked...


  • Kolkata, West Bengal, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world's biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We're harnessing the power of technology and humanity to create...

  • Data Engineer-GGN

    1 week ago


    Kolkata, West Bengal, India Programming Full time

    Job Title:Data Engineer at Company XYZAre you passionate about working with data? Join our team as a Data Engineer! We are looking for someone with solid experience (3-6 years) in Python, Apache Spark, and Databricks, plus a flair for machine learning.Key Responsibilities:Develop and maintain scalable data pipelines using Python and Apache Spark.Collaborate...


  • Kolkata, West Bengal, India CliniSys Full time

    Essential FunctionsExtract data from on-prem databases using appropriate tools and techniques.Transform and clean the data to ensure its suitability for PowerBI analysis.Build and maintain data pipelines for regular data refresh in PowerBI.Monitor and maintain Azure Synapse pipelines for on-prem data extraction and transformation.Collaborate with analysts to...


  • Kolkata, West Bengal, India CliniSys Full time

    Essential Functions Extract data from on-prem databases using appropriate tools and techniques. Transform and clean the data to ensure its suitability for PowerBI analysis. Build and maintain data pipelines for regular data refresh in PowerBI. Monitor and maintain Azure Synapse pipelines for on-prem data extraction and transformation. Collaborate...

  • Data Engineer

    1 week ago


    Kolkata, West Bengal, India Randstad India Full time

    One of our esteemed clients is looking for a Data Engineer in Bangalore. Responsibilities: Design and automate crucial data pipelines to seamlessly integrate with downstream analytics and production systems. Collaborate with cross-functional teams to incorporate new functionalities into existing data pipelines, including lower test environments for...

  • Data Engineer

    3 weeks ago


    Kolkata, West Bengal, India CA-One India Full time

    Job Title : Data : Kolkata, IndiaResponsibilities :- Design, build, and maintain scalable data pipelines.- Develop SparkSQL queries and BigQuery ML models.- Integrate Kafka for real-time data processing.- Manage BigQuery Data Transfer Service and Snowflake SQL queries.- Collaborate with cross-functional teams on data :- Bachelor's degree in Computer...

  • Data Engineer

    2 months ago


    Kolkata, West Bengal, India CA-One India Full time

    Job Title : Data : Kolkata, IndiaResponsibilities :- Design, build, and maintain scalable data pipelines.- Develop SparkSQL queries and BigQuery ML models.- Integrate Kafka for real-time data processing.- Manage BigQuery Data Transfer Service and Snowflake SQL queries.- Collaborate with cross-functional teams on data :- Bachelor's degree in Computer...


  • Kolkata, West Bengal, India timesjobs Full time

    Hiring For Data Engineer- Databricks Azure Data Factory Python Job Description Data Engineer- Databricks Azure Data Factory Python Desired Profile Skills: Azure Databricks Azure Data Factory Python, implementing both structured and unstructured data, Data migration solutions, Data Warehousing, Data processing / ETL / Big data, Hadoop, Java, Spark, Scala, and...


  • Kolkata, West Bengal, India timesjobs Full time

    Hiring For Data Engineer- Databricks Azure Data Factory Python Job Description Data Engineer- Databricks Azure Data Factory Python Desired Profile Skills: Azure Databricks Azure Data Factory Python, implementing both structured and unstructured data, Data migration solutions, Data Warehousing, Data processing / ETL / Big data, Hadoop, Java, Spark, Scala, and...


  • Kolkata, West Bengal, India Techylla Full time

    Company Overview Techylla is a specialized IT consulting firm based in India and US. Our focus lies in collaborating with Life Sciences companies, offering Data-Driven Decision Making solutions that encompass a spectrum of IT services and technology-driven business solutions. Our vision is to become the preferred partner for analytics, bridging the gap...


  • Kolkata, West Bengal, India NTT Full time

    JOB DESCRIPTION NTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future. In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our...


  • Kolkata, West Bengal, India NTT Full time

    JOB DESCRIPTIONNTT is a leading global IT solutions and services organisation that brings together people, data and things to create a better and more sustainable future.In today's 'iNTTerconnected' world, connections matter more now than ever. By bringing together talented people, world-class technology partners and emerging innovators, we help our clients...