1,101 Etl Development jobs in India

Data Transformation - Associate

Mumbai, Maharashtra JPMorgan Chase & Co.

Posted today

Job Viewed

Tap Again To Close

Job Description

Join our team as a Data Transformation Associate, where you'll design scalable data pipelines and transform data to support analytics and business intelligence. Promote decisions impacting product design and technical operations, while developing AI/ML solutions on AWS to tackle complex business challenges and foster innovation.

As a Data Transformation Associate within our Data Transformation team, you will be responsible for designing, developing, and maintaining scalable data pipelines, transforming data to a Logical Data Model (LDM) to support data analytics and business intelligence initiatives. You will promote decisions that influence product design, application functionality, and technical operations and processes, and collaborate with cross-functional teams to understand business requirements and formulate technical solutions. You will also design, develop, and deploy AI/ML solutions on AWS to solve complex business problems and promote innovation.

Job responsibilities:

  • Design, develop, and maintain scalable data pipelines to transform data for analytics and business intelligence.
  • Drive decisions that influence product design, application functionality, and technical operations.
  • Design, develop, and deploy AI/ML solutions on AWS to address complex business problems and foster innovation.
  • Collaborate with cross-functional teams to understand business requirements and formulate technical solutions.
  • Manage business stakeholders with product roadmap and delivery.
  • Required qualifications, capabilities, and skills:

  • Experience with data platforms such as Databricks, Snowflake, and Redshift.
  • Familiarity with cloud platforms like AWS, Azure, or Google Cloud.
  • Proficiency in programming languages such as Enterprise Java or Python.
  • Experience with technologies like Spark and Kafka.
  • Experience using machine learning frameworks like TensorFlow or PyTorch.
  • Ability to design and implement AI/ML models and algorithms to address business challenges and opportunities.
  • Develop and deploy scalable machine learning solutions. Optimize and fine-tune machine learning models for performance and scalability.
  • Ensure the security, reliability, and efficiency of AI/ML solutions deployed on AWS.
  • Conduct research and experiments to explore new AI/ML techniques and methodologies.
  • Good understanding of data privacy, security best practices, and compliance requirements to ensure safe and ethical use of LLMs.
  • This advertiser has chosen not to accept applicants from your region.

    Data Transformation Architect

    Hyderabad, Andhra Pradesh Talent Worx

    Posted today

    Job Viewed

    Tap Again To Close

    Job Description

    Talworx is an emerging recruitment consulting and service firm, we are hiring for our client which provides governments, businesses, and individuals with market data, expertise, and technology solutions for confident decision-making. Our services span from global energy solutions to sustainable finance solutions. From helping our customers perform investment analysis to guiding them through sustainability and energy transition across supply chains, our solutions help unlock new opportunities and solve challenges.

    Location:  Hybrid work from office from nearest client branch, Preferrably from Hyderabad, Bangalore, Gurgaon.

    The Team:  You will be part of a rapidly growing organization, joining the team of highly motivated and professional Data Scientists and Machine Learning Engineers within Market Intelligence division. Market Intelligence provides financial and industry data, research, news and analytics to investment professionals, government agencies, corporations, and universities worldwide. We integrate news, comprehensive market and sector-specific data and analytics into a variety of tools to help clients track performance, generate alpha, identify investment ideas, understand competitive and industry dynamics, perform valuations, and assess credit risk.

    The Impact:  We are looking for candidates that are passionate to architect, build, scale and deploy Machine Learning Models and their associated Software Technology components that provide timely and essential intelligence to our disparate array of customers across the globe.

    What’s in it for you:  We provide a highly inclusive work environment where in you can bring your whole self to work to assist us in achieving its mission of being one of the leading providers of the highest quality risk evaluations and analytical information to the world’s financial markets. As an integral part of our team, you will be working on cutting edge state-of-the-art technology stack. In this role, you will work on multiple data science projects in collaboration with internal and external project owners on the product, commercial, and data team. You will be responsible for providing machine learning engineers support, create data pipeline for modeling, scale models, develop APIs to help move machine learning models in production. You will collaborate with the data scientists and production-oriented software engineers

    .

    Responsibilities Responsibilities:

    • Construct machine learning lifecycle management including data collection, normalization, and standardization within a data pipeline construction
    • Architect technology components and build applications and interfaces for customer's consumption
    • Experiment, develop and productionize high quality machine learning services and platforms to make huge technology and business impact.
    • Develop hosting platform for machine learning models.
    • Create pipelines to query and retrieve and update data for existing applications to keep them updated
    • Supervise the scaling and management of the machine learning modeling ecosystem
    • Work alongside data scientists and product owners to improve aspects of their lines of business through machine learning

    What We’re Looking For:

    • Extensive problem-solving ability in designing complex data and cloud architectures,
    • Must possess strong hands on experience with Spark, AWS Bigdata (Hadoop), Python and pyspark
    • Possess excellent verbal & written communication skills
    • Expertise in application, data, and infrastructure architecture disciplines
    • Advanced knowledge of architecture and design across all systems
    • Proficiency in multiple machine learning programming languages including Python, PySpark, or Scala
    • 5 + years of experience with big-data technologies such as Hadoop, Spark, SparkML, etc
    • Able to understand various data structures and common methods in data transformation
    • Knowledge of industry-wide technology trends and best practices
    • Ability to work in large, collaborative teams to achieve organizational goals
    • Passionate about building an innovative culture
    • Familiarity with MLOps and ModelOps
    • Docker, Kubernetes, AWS, Python
    • Familiarity with or clear interest in learning about financial Markets.
    This advertiser has chosen not to accept applicants from your region.

    Senior Data Transformation Analyst

    Bangalore, Karnataka S&P Global

    Posted 3 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    **About the Role:**
    **Grade Level (for internal use):**
    09
    **Job Summary:**
    We are seeking an experienced Oil and Gas Operations Specialist with a minimum of 5+ years in oil and gas operations. The ideal candidate will be well-versed in industry terminology and able to collaborate effectively with both onshore and offshore teams. This role requires a proactive approach to managing operations, ensuring data quality, and providing exceptional client support.
    **Key Responsibilities:**
    + Collaborate closely with the manager to understand leadership priorities and align operational plans accordingly.
    + Work with the Data Transformation team to enhance data quality by providing technical oversight on data entered by vendor partners.
    + Develop a strong understanding of Service Level Agreements (SLAs) and metrics, ensuring quality control and operational oversight of vendor operations.
    + Address client issues on a daily basis, providing timely resolutions and participating in client calls.
    + Serve as the subject matter expert (SME) for oil and gas operations, becoming the "Go-To" person for relevant states.
    + Build expertise in S&P Global's internal systems and processes, as well as client-facing products related to US wells and production.
    + Familiarity with SQL and process automation is a plus.
    **Qualifications:**
    + Minimum of 5 years of experience in oil and gas operations.
    + Strong understanding of oil and gas terminology.
    + Excellent communication and interpersonal skills.
    + Ability to work collaboratively in a team environment.
    + Background in SQL and process automation is advantageous.
    **What's In It For** **You?**
    **Our Purpose:**
    Progress is not a self-starter. It requires a catalyst to be set in motion. Information, imagination, people, technology-the right combination can unlock possibility and change the world.
    Our world is in transition and getting more complex by the day. We push past expected observations and seek out new levels of understanding so that we can help companies, governments and individuals make an impact on tomorrow. At S&P Global we transform data into Essential Intelligence®, pinpointing risks and opening possibilities. We Accelerate Progress.
    **Our People:**
    We're more than 35,000 strong worldwide-so we're able to understand nuances while having a broad perspective. Our team is driven by curiosity and a shared belief that Essential Intelligence can help build a more prosperous future for us all.
    From finding new ways to measure sustainability to analyzing energy transition across the supply chain to building workflow solutions that make it easy to tap into insight and apply it. We are changing the way people see things and empowering them to make an impact on the world we live in. We're committed to a more equitable future and to helping our customers find new, sustainable ways of doing business. We're constantly seeking new solutions that have progress in mind. Join us and help create the critical insights that truly make a difference.
    **Our Values:**
    **Integrity, Discovery, Partnership**
    At S&P Global, we focus on Powering Global Markets. Throughout our history, the world's leading organizations have relied on us for the Essential Intelligence they need to make confident decisions about the road ahead. We start with a foundation of **integrity** in all we do, bring a spirit of **discovery** to our work, and collaborate in close **partnership** with each other and our customers to achieve shared goals.
    **Benefits:**
    We take care of you, so you can take care of business. We care about our people. That's why we provide everything you-and your career-need to thrive at S&P Global.
    Our benefits include:
    + Health & Wellness: Health care coverage designed for the mind and body.
    + Flexible Downtime: Generous time off helps keep you energized for your time on.
    + Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.
    + Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.
    + Family Friendly Perks: It's not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families.
    + Beyond the Basics: From retail discounts to referral incentive awards-small perks can make a big difference.
    For more information on benefits by country visit: Hiring and Opportunity at S&P Global:**
    At S&P Global, we are committed to fostering a connected and engaged workplace where all individuals have access to opportunities based on their skills, experience, and contributions. Our hiring practices emphasize fairness, transparency, and merit, ensuring that we attract and retain top talent. By valuing different perspectives and promoting a culture of respect and collaboration, we drive innovation and power global markets.
    **Recruitment Fraud Alert:**
    If you receive an email from a spglobalind.com domain or any other regionally based domains, it is a scam and should be reported to . S&P Global never requires any candidate to pay money for job applications, interviews, offer letters, "pre-employment training" or for equipment/delivery of equipment. Stay informed and protect yourself from recruitment fraud by reviewing our guidelines, fraudulent domains, and how to report suspicious activity here ( .
    ---
    **Equal Opportunity Employer**
    S&P Global is an equal opportunity employer and all qualified candidates will receive consideration for employment without regard to race/ethnicity, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, marital status, military veteran status, unemployment status, or any other status protected by law. Only electronic job submissions will be considered for employment.
    If you need an accommodation during the application process due to a disability, please send an email to:   and your request will be forwarded to the appropriate person. 
    **US Candidates Only:** The EEO is the Law Poster   describes discrimination protections under federal law. Pay Transparency Nondiscrimination Provision - - Professional (EEO-2 Job Categories-United States of America), DTMGOP202.1 - Middle Professional Tier I (EEO Job Group), SWP Priority - Ratings - (Strategic Workforce Planning)
    **Job ID:** 315200
    **Posted On:** 2025-08-06
    **Location:** Bangalore, Karnataka, India
    This advertiser has chosen not to accept applicants from your region.

    Senior Engineer - Data transformation

    Bengaluru, Karnataka Invenco by GVR

    Posted 4 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    Join our global data team as a Senior Data Engineer to shape our data landscape and drive advanced analytics capabilities across the organization.

    What You Will Do (Job Responsibilities)
    This position will be involved in:
    • Establishing our data transformation capability. Create Semantic Data Models as the foundation for self-service enterprise reporting
    • Implementing modern data transformation tools (DBT Cloud, Coalesce etc)
    • Transformation of raw data into presentation layer EDW structure
    • Collaborate & work closely with analysts and stakeholders
    • Innovate & adapt new technologies to improve our data infrastructure
    • Optimize existing processes for performance and scalability
    • Takes ownership of problems and proactively resolves technical problems, ensuring that technical solutions continue to meet business requirements and aligns with the vision, mission, objectives, strategy of the team highly motivated, goal driven, innovative, curious and open minded, Fun to work with, team player

    )Educational Requirements :
    We are looking for someone with the following experience:
    • Qualification in Computer Science or Information Systems, Engineering or related field is preferred
    • 5+ years of experience in a data engineering role with a focus on data transformation
    • Strong understanding of data modeling, ETL concepts, and data warehouse design principles
    • Experience with cloud data platforms (e.g., AWS, GCP, Azure) and their data services. Snowflake experience preferred
    • Proven experience with DBT/Coalesce or similar modern data transformation tools, including advanced features for complex data transformation projects. DBT Cloud experience preferred
    • Expert level skills in SQL and other Snowflake relevant languages such as Python
    • Experience with GIT and CI/CD practices
    • Excellent problem-solving skills and attention to detail
    • Experience with common modern ingestion tools
    • Experience with visualization tools like Power BI or equivalent
    • Understanding of typical ERP/CRM system data structures
    • Strong communication & collaboration skills, with the ability to work effectively in a remote team environment in an agile environment
    • Ability to provide technical leadership experience to junior team members
    This advertiser has chosen not to accept applicants from your region.

    Logistics Data Transformation Developer

    Bengaluru, Karnataka Confidential

    Posted today

    Job Viewed

    Tap Again To Close

    Job Description

    As a Logistics Data Transformation Developer, you will work on a wide variety of Business Transformation projects. The role would entail:

    • Understanding the business transformation needs and its impact on the SAP ERP data.
    • Participating in customer workshops, gathering and analyzing data requirements.
    • Defining the data rules and implementation.
    • Active engagement in the entire data transformation/migration cycle.
    • Ownership of the module aligned to.
    • Communication and collaboration with internal and external project teams.

    Your Profile

    Ideally in-depth knowledge, and skills in the following:

    • 5 to 8+ years of experience in implementation / support projects/
    • migration projects in SAP and ability to write basic to medium level ABAP programs.
    • Good to have experience in SAP data migration projects.
    • Minimum 2+ years of experience working in SAP Logistics (SD/MM/PP & QM) any of these modules
    • Good understanding of the master- and transactional data from functional/process perspective.
    • Ability to debug the ABAP code, troubleshoot and fix basic code.
    • Knowledge of cross functional integration points.
    • Very good analytical and communication skills.
    • A self-starter and a team player

    Skills Required
    Sap, Pp, Abap, Troubleshoot, Collaboration
    This advertiser has chosen not to accept applicants from your region.

    Associate,Data Scientist,Transformation & Data

    Mumbai, Maharashtra DBS Bank

    Posted today

    Job Viewed

    Tap Again To Close

    Job Description

    Associate,Data Scientist,Transformation & Data-(WD76198)

    Business Function : 

    Group Strategic Marketing & Communications is responsible for building, protecting, and enhancing the DBS brand in an increasingly commoditised banking industry. Working closely with all units within the DBS Group, we create and execute integrated strategies that focus on communications especially digital communications, marketing and research to differentiate – and elevate – our brand from the rest of the competition. 

    Key Accountabilities :-

    Responsibilities  

  • Involve in building Machine Learning models , data science solutions and working with state-of-art data science and Machine Learning Tools such as Spark, Sklearn, Tensorflow , and will have mastery in any number of Analytic Programming languages /platforms like Python, R . etc.
  • Work with large datasets, distributed Big Data Platforms for a variety of advanced analytics solutions across the bank ranging from recommendation engines, propensity models, customer segmentation, Graph models, Credit underwriting model, pricing and more
  • Engage with business and product teams on the problem formulation and create solution framework
  • End-to-End execution from data collection, analysis, feature engineering, model development, implementation (including testing) and monitoring
  • Design A/B test experiments to test the model effectiveness and Analyze to identify the target segments
  • Drive the testing and deployment of the machine learning products into production
  • Excellent pattern recognition and predictive modelling skills
  • Understand the Product Journeys and help business to improve the customer experience
  • Requirements

    Non-technical Experience

  • Strong academic background
  • Should be self-motivated and adaptable.
  • Should possess a creative bent of mind and should be a strategic thinker.
  • Should be focused and well organized
  • Technical Experience

  • 3-12 years of experience in Data Science/Analytics (consumer banking, ecommerce, retail, telecoms, technology) with demonstrated track of generating value through DS/ML/AI solutions.
  • Should have full understanding of applications of DS/ML/AI solutions to various business problems.
  • Proficiency in using databases like Teradata, NoSQL and Hadoop using SQL, Hive, Spark, etc.,
  • Programming experience in Python, R, Spark, SAS, TensorFlow or other machine learning oriented programming languages/platforms.
  • Good understanding of technology tools especially those related to analytics, data & modelling.
  • Ability to communicate complex analysis/models across a diverse team.
  • Good written and oral communication skills.
  • Education / Preferred Experience:-

  • B.Sc./ M.Sc. or equivalent degree in Statistics, Analytics, Applied Mathematics, Operation Research, or equivalent quantitative fields preferred.
  • Prior Working experience of AI / ML / DS is must
  • -en

    Primary Location

    :India-Maharashtra-Mumbai

    Job

    :Analytics

    Schedule

    :Regular

    Employee Status

    :Full-time:

    Job Posting

    :Aug 12, 2025, 2:30:00 AM
    This advertiser has chosen not to accept applicants from your region.

    Data Collection Transformation

    Mumbai, Maharashtra MSCI Inc

    Posted today

    Job Viewed

    Tap Again To Close

    Job Description

    **Data**:
    Bring clarity to our clients’ investment decisions by collecting, processing and distributing industry-leading data.

    **Sustainable Investing**:
    Shape the future of Sustainable Investing

    **Are you curiously minded and passionate about creating value that really matters?**

    MSCI is dedicated to providing sustainability and climate data and tools to support investment decision making, and today we are proud to work with around 3,000 clients worldwide.

    **Your Team Responsibilities**: MSCI ESG Data collection team is involved in acquisition of ESG data at scale and QA of the collected data and responsible for establishing and maintaining highest level of data quality and standards across all datasets feeding our ESG products. As an Data Collection Transformation Senior Associate, you will be responsible for leading and delivering on several initiatives as part of the ESG transformation agenda, to support rapidly evolving ESG landscape and its adoption in the financial market.
    - ** Your Key Responsibilities**: - As a member of MSCI Data Acquisition and Collection team, you are expected to have a strong interest in general Environment, Social, Governance, Climate and policy frameworks around these domains as well as regulatory trends - Take active part in projects dealing with “electronification” of ESG & Climate frameworks and principles into data definitions which can be operationalized for collection - Collaborate with Research teams on building data collection templates and with technology teams to translate these into implementable data models - Do hands-on research with new data sets by studying company disclosures to help connect research proposals with implementable solution which are scalable - Independently run analysis on data sets (either collected or from third party) to detect trends/patterns (EDA) and propose ways to build anomaly detection on new and existing content - Analyze & research the historical data corrections across all ESG & Climate data and propose & implement contextual/thematic QA to detect cases that potentially may not be captured in current QA framework - “Codify” data definitions with an intent to build NLP driven data extraction models (leveraging Traditional approaches/LLMs) to automate detection and extraction of “Facts” from company disclosures - Help design and set-up new data collection processes and help with integration of these processes with ongoing Data - Deliver top quality data aligned with MSCI methodology, service level agreements, and regulatory requirements;
    - Steer to improve methodology and SOP documents leveraging data and content expertise;
    - Drive process improvements to ensure consistent data quality and efficiency, such as automation of data quality diagnostics by developing a new system/tool that will enable quality assessment of data without manual intervention;
    - Contribute to process improvements to ensure consistent data quality and efficiency, such as automation of data quality diagnostics by developing a new system/tool which will enable quality assessment of data without manual intervention;
    - Work with internal stakeholders and downstream teams on understanding data requirement, data QC scope and data delivery;
    - Create reports/dashboards which provide quantitative data assessment metrics which justify recommendations. Visualization, outlier detection/analysis, data summaries, etc. - Sharing plans, recommendations, summaries with management through conference calls, meetings and presentations with internal/external teams, Research and product
    - ** About MSCI**:
    **What we offer you**
    - Transparent compensation schemes and comprehensive employee benefits, tailored to your location, ensuring your financial security, health, and overall wellbeing.
    - Flexible working arrangements, advanced technology, and collaborative workspaces.
    - A culture of high performance and innovation where we experiment with new ideas and take responsibility for achieving results.
    - A global network of talented colleagues, who inspire, support, and share their expertise to innovate and deliver for our clients.
    - Multi-directional career paths that offer professional growth and development through new challenges, internal mobility and expanded roles.
    - We actively nurture an environment that builds a sense of inclusion belonging and connection, including eight Employee Resource Groups. All Abilities, Asian Support Network, Black Leadership Network, Climate Action Network, Hola! MSCI, Pride & Allies, Women in Tech, and Women’s Leadership Forum.

    At MSCI we are passionate about what we do, and we are inspired by our purpose - to power better investment decisions. You’ll be part of an industry-leading network of creative, curious, and entrepreneurial pioneers. This is a space where you can challenge yourself, set new standards and perform beyond expectations for yourself, our clients, and our industry.

    MSCI is a leading provider of critical decision
    This advertiser has chosen not to accept applicants from your region.
    Be The First To Know

    About the latest Etl development Jobs in India !

    Data Pipeline Irc172490

    Nagpur, Maharashtra GlobalLogic

    Posted today

    Job Viewed

    Tap Again To Close

    Job Description

    Job:
    - IRC172490- Location:
    - India - Nagpur- Designation:
    - Consultant- Experience:
    - 5-10 years- Function:
    - Engineering**Description**:
    seeking an experienced data pipelines engineer for our data engineering organization. As a member of our team, you will develop highly-performant, scalable and available data pipelines for bp internal
    - and external-facing business domains. You must be able to understand business data requirements, be familiar with cloud-based data pipelining tools and thrive working in a fast-paced environment.

    **Requirements**:

    - 3+ years of hands-on software development, data engineering, and systems architecture
    - 2+ years of experience in designing, developing and maintaining high performance, low latency and large-scale data pipelines.
    - 2+ years of experience in SQL optimization and performance tuning, and development experience in programming languages like Python, PySpark, Scala etc.)
    - 2+ years in cloud data engineering experience in Azure or/and AWS
    - Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
    - Experience with data modeling, data warehousing, and building high-volume ETL/ELT pipelines
    - Experience with modern data technologies such as SQL and non-SQL databases, REST / GraphQL APIs, data event streaming technologies, and in-memory data storage.
    - Experience with version control systems like Github and deployment & CI/CD tools
    - Excellent communication skills, both verbal and written

    **Preferences**:
    data pipeline

    **Responsibilities**:

    - As a member of our team, you will design, develop, test and maintain highly performant, scalable, robust and available data pipelines for our business teams.
    - Work closely with the data science and engineering team and the data platform team to implement automated data pipeline workflows in the Azure/AWS cloud platforms to enable data ingestion, processing, and distribution.
    - Deploy scalable solutions typical to critical bp business environments.
    - Build the automation and monitoring frameworks that capture metrics and operational KPIs for data pipeline quality and performance.
    - Build prototypes and workflows to try out new ideas in an iterative manner

    **What We Offer**:
    **Exciting Projects**: We focus on industries like High-Tech, communication, media, healthcare, retail and telecom. Our customer list is full of fantastic global brands and leaders who love what we build for them.

    **Collaborative Environment**: You Can expand your skills by collaborating with a diverse team of highly talented people in an open, laidback environment — or even abroad in one of our global centers or client facilities!

    **Work-Life Balance**: GlobalLogic prioritizes work-life balance, which is why we offer flexible work schedules, opportunities to work from home, and paid time off and holidays.

    **Professional Development**: Our dedicated Learning & Development team regularly organizes Communication skills training(GL Vantage, Toast Master),Stress Management program, professional certifications, and technical and soft skill trainings.

    **Excellent Benefits**: We provide our employees with competitive salaries, family medical insurance, Group Term Life Insurance, Group Personal Accident Insurance, NPS(National Pension Scheme ), Periodic health awareness program, extended maternity leave, annual performance bonuses, and referral bonuses.

    **Fun Perks**: We want you to love where you work, which is why we host sports events, cultural activities, offer food on subsidies rates, Corporate parties. Our vibrant offices also include dedicated GL Zones, rooftop decks and GL Club where you can drink coffee or tea with your colleagues over a game of table and offer discounts for popular stores and restaurants!

    **About GlobalLogic**: GlobalLogic is a leader in digital engineering. We help brands across the globe design and build innovative products, platforms, and digital experiences for the modern world. By integrating experience design, complex engineering, and data expertise—we help our clients imagine what’s possible, and accelerate their transition into tomorrow’s digital businesses. Headquartered in Silicon Valley, GlobalLogic operates design studios and engineering centers around the world, extending our deep expertise to customers in the automotive, communications, financial services, healthcare and life sciences, manufacturing, media and entertainment, semiconductor, and technology industries. GlobalLogic is a Hitachi Group Company operating under Hitachi, Ltd. (TSE: 6501) which contributes to a sustainable society with a higher quality of life by driving innovation through data and technology as the Social Innovation Business.
    This advertiser has chosen not to accept applicants from your region.

    Cyber Data Pipeline Engineer

    Bengaluru, Karnataka Dexian India

    Posted today

    Job Viewed

    Tap Again To Close

    Job Description

    Overview

    We are seeking a skilled and motivated Data Pipeline Engineer to join our team. In this role, you will manage and maintain critical data pipeline platforms that collect, transform, and transmit cyber events data to downstream platforms, such as ElasticSearch and Splunk. You will be responsible for ensuring the reliability, scalability, and performance of the pipeline infrastructure while building complex integrations with cloud and on-premises cyber systems.


    Our key stakeholders are cyber teams including security response, investigations and insider threat.

    Role Profile

    A successful applicant will contribute to several important initiatives including:

    • Collaborate with Cyber teams to identify, onboard, and integrate new data sources into the platform.
    • Design and implement data mapping, transformation, and routing processes to meet analytics and monitoring requirements.
    • Developing automation tools that integrate with in-house developed configuration management frameworks and APIs
    • Monitor the health and performance of the data pipeline infrastructure.
    • Working as a top-level escalation point to perform complex troubleshoots, working with other infrastructure teams to resolve issues
    • Create and maintain detailed documentation for pipeline architecture, processes, and integrations.

    Required Skills

    • Hands-on experience deploying and managing large-scale dataflow products like Cribl, Logstash or Apache NiFi
    • Hands-on experience integrating data pipelines with cloud platforms (e.g., AWS, Azure, Google Cloud) and on-premises systems.
    • Hands-on experience in developing and validating field extraction using regular expressions.
    • A solid understanding of Operating Systems and Networking concepts: Linux/Unix system administration, HTTP and encryption.
    • Good understanding of software version control, deployment & build tools using DevOps SDLC practices (Git, Jenkins, Jira)
    • Strong analytical and troubleshooting skills
    • Excellent verbal & written communication skills
    • Appreciation of Agile methodologies, specifically Kanban

    Desired Skills

    • Enterprise experience with a distributed event streaming platform like Apache Kafka, AWS Kinesis, Google Pub/Sub, MQ
    • Infrastructure automation and integration experience, ideally using Python and Ansible
    • Familiarity with cybersecurity concepts, event types, and monitoring requirements.
    • Experience in Parsing and Normalizing data in Elasticsearch using Elastic Common Schema (ECS)
    This advertiser has chosen not to accept applicants from your region.

    Cyber Data Pipeline Engineer

    Bengaluru, Karnataka Dexian India

    Posted 4 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    Title: Cyber Data Pipeline Engineer

    Location : Bengaluru

    Experience: 7 to 14 years

    Role Profile

    A successful applicant will contribute to several important initiatives including:

    • Collaborate with Cyber teams to identify, onboard, and integrate new data sources into the platform.
    • Design and implement data mapping, transformation, and routing processes to meet analytics and monitoring requirements.
    • Developing automation tools that integrate with in-house developed configuration management frameworks and APIs
    • Monitor the health and performance of the data pipeline infrastructure.
    • Working as a top-level escalation point to perform complex troubleshoots, working with other infrastructure teams to resolve issues
    • Create and maintain detailed documentation for pipeline architecture, processes, and integrations.

    Required Skills

    • Hands-on experience deploying and managing large-scale dataflow products like Cribl, Logstash or Apache NiFi
    • Hands-on experience integrating data pipelines with cloud platforms (e.g., AWS, Azure, Google Cloud) and on-premises systems.
    • Hands-on experience in developing and validating field extraction using regular expressions.
    • A solid understanding of Operating Systems and Networking concepts: Linux/Unix system administration, HTTP and encryption.
    • Good understanding of software version control, deployment & build tools using DevOps SDLC practices (Git, Jenkins, Jira)
    • Strong analytical and troubleshooting skills
    • Excellent verbal & written communication skills
    • Appreciation of Agile methodologies, specifically Kanban

    Desired Skills

    • Enterprise experience with a distributed event streaming platform like Apache Kafka, AWS Kinesis, Google Pub/Sub, MQ
    • Infrastructure automation and integration experience, ideally using Python and Ansible
    • Familiarity with cybersecurity concepts, event types, and monitoring requirements.
    • Experience in Parsing and Normalizing data in Elasticsearch using Elastic Common Schema (ECS)
    This advertiser has chosen not to accept applicants from your region.
     

    Nearby Locations

    Other Jobs Near Me

    Industry

    1. request_quote Accounting
    2. work Administrative
    3. eco Agriculture Forestry
    4. smart_toy AI & Emerging Technologies
    5. school Apprenticeships & Trainee
    6. apartment Architecture
    7. palette Arts & Entertainment
    8. directions_car Automotive
    9. flight_takeoff Aviation
    10. account_balance Banking & Finance
    11. local_florist Beauty & Wellness
    12. restaurant Catering
    13. volunteer_activism Charity & Voluntary
    14. science Chemical Engineering
    15. child_friendly Childcare
    16. foundation Civil Engineering
    17. clean_hands Cleaning & Sanitation
    18. diversity_3 Community & Social Care
    19. construction Construction
    20. brush Creative & Digital
    21. currency_bitcoin Crypto & Blockchain
    22. support_agent Customer Service & Helpdesk
    23. medical_services Dental
    24. medical_services Driving & Transport
    25. medical_services E Commerce & Social Media
    26. school Education & Teaching
    27. electrical_services Electrical Engineering
    28. bolt Energy
    29. local_mall Fmcg
    30. gavel Government & Non Profit
    31. emoji_events Graduate
    32. health_and_safety Healthcare
    33. beach_access Hospitality & Tourism
    34. groups Human Resources
    35. precision_manufacturing Industrial Engineering
    36. security Information Security
    37. handyman Installation & Maintenance
    38. policy Insurance
    39. code IT & Software
    40. gavel Legal
    41. sports_soccer Leisure & Sports
    42. inventory_2 Logistics & Warehousing
    43. supervisor_account Management
    44. supervisor_account Management Consultancy
    45. supervisor_account Manufacturing & Production
    46. campaign Marketing
    47. build Mechanical Engineering
    48. perm_media Media & PR
    49. local_hospital Medical
    50. local_hospital Military & Public Safety
    51. local_hospital Mining
    52. medical_services Nursing
    53. local_gas_station Oil & Gas
    54. biotech Pharmaceutical
    55. checklist_rtl Project Management
    56. shopping_bag Purchasing
    57. home_work Real Estate
    58. person_search Recruitment Consultancy
    59. store Retail
    60. point_of_sale Sales
    61. science Scientific Research & Development
    62. wifi Telecoms
    63. psychology Therapy
    64. pets Veterinary
    View All Etl Development Jobs