3 Hdfs jobs in India

Principal Engineer – Ozone/HDFS

Bengaluru, Karnataka Cloudera

Posted today

Job Viewed

Tap Again To Close

Job Description

Job Description

At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises.

Cloudera is looking for an exceptional and passionate software engineer with a strong distributed systems background to join the Storage Engineering team focused on building Apache Ozone. The Storage team is responsible for primary storage and storage access layers, which are core to the platform. Apache Ozone () provides a massively scalable distributed object store with a distributed file system interface. Ozone is designed to scale to tens of billions of files and blocks, and overcome the limitations of Hadoop Distributed File System (HDFS), namely, millions of small files and managing a huge number of datanodes. 

Opportunity to join the team that created and wrote most of the HDFS code and make a huge impact on the big data and cloud computing industry.

As a Principal Software Engineer, you will:

  • You will be directly involved in the design and implementation of the core feature set of Apache Ozone and Apache Ratis (open-source RAFT implementation) 

  • You will regularly contribute code and design docs to the Apache open-source community.

  • As part of storage engineering, you will support enterprise customers running 100s of petabytes-scale big data analytics and ML/AI pipelines. 

  • You will partner with Engineering leaders, product managers, and cross-functional teams as a part of the Cloudera Data platform ecosystem in understanding requirements and turning them into a solid design and implementation, and facilitating integration and adoption.

  • Additionally, in this role, you will be responsible for leading a talented group of engineers working on the feature and mentoring junior engineers.

  • We are excited about you if you have:

  • BS, MS, or PhD in Computer Science

  • Bachelor's +15 , Master's +12 years of relevant industry experience required (8+ for PhD candidate)

  • Strong backend engineering skill set with expertise in Java, or strong C++ skills, with intermediate Java expertise

  • Passionate about programming. Clean coding habits, attention to detail, and focus on quality

  • Experience with large-scale, distributed systems design and development with a strong understanding of scaling, replication, consistency, and high availability

  • Solid experience with system software design and development with a strong understanding of computer architecture, storage, network, and IO subsystems, and distributed systems

  • Hands-on programmer with strong data structures and algorithms skillset

  • Strong oral and written communication skills

  • You may also have:

  • Strong background in a distributed storage system, including file systems, database storage internals, NoSQL storage, or distributed hash tables

  • Strong background in performance tuning, identifying performance bottlenecks, and implementing performance optimizations

  • Strong understanding of the Apache Big Data ecosystem and over 3+ years of experience in systems software, including file systems

  • Recognized contributions to open source projects

  • Experience using projects such as Hive, Pig, MapReduce, HBase, etc., is a big plus

  • Good Understanding of storage development, RAFT replication framework, or equivalent distributed consensus frameworks

  • What you can expect from us:

  • Generous PTO Policy 

  • Support work life balance with

  • Flexible WFH Policy 

  • Mental & Physical Wellness programs 

  • Phone and Internet Reimbursement program 

  • Access to Continued Career Development 

  • Comprehensive Benefits and Competitive Packages 

  • Employee Resource Groups

  • This advertiser has chosen not to accept applicants from your region.

    Data Analyst (SQL,HDFS, Hive)

    Bengaluru, Karnataka RiskInsight Consulting Pvt Ltd

    Posted today

    Job Viewed

    Tap Again To Close

    Job Description

    RiskInsight Consulting Pvt Ltd is looking for a Data Analyst with expertise in SQL and HDFS (Hadoop Distributed File System) to join our dynamic team. The selected candidate will play a crucial role in analyzing large datasets and providing actionable insights to drive business decisions. If you are a detail-oriented individual with a passion for data, we want to hear from you!

    1. Design, implement, and maintain data pipelines and workflows using HDFS for data management.
    2. Conduct data extraction, transformation, and loading (ETL) processes using SQL and Hive.
    3. Perform rigorous data analysis to identify trends, inconsistencies, and insights for stakeholders.
    4. Collaborate with business units to understand their analytical needs and deliver data-driven insights.
    5. Create visualizations and generate reports to present findings to both technical and non-technical audiences.
    6. Ensure data integrity and security throughout all stages of data processing.
    7. Stay informed about industry advancements in data analysis and contribute to best practices in the field.

    Requirements

    1. Bachelor’s degree in Computer Science, Data Science, Mathematics, or related field.
    2. Proven experience in a Data Analyst role with strong expertise in SQL and HDFS.
    3. Hands-on experience with Hive for querying and analyzing large datasets.
    4. Familiarity with data visualization tools such as Tableau, Power BI, or similar.
    5. Excellent problem-solving and analytical skills with the ability to draw meaningful conclusions from data.
    6. Strong communication skills to effectively share insights with diverse teams.
    7. Ability to work independently and as part of a team in a fast-paced environment.

    Benefits

    Competitive salary and benefits package.

    Opportunity to work on cutting-edge technologies and solve complex challenges.

    Dynamic and collaborative work environment with opportunities for growth and career advancement.

    Regular training and professional development opportunities.

    This advertiser has chosen not to accept applicants from your region.

    Java, Spark - Lead Developer (Java, Spark, HDFS, Hive, Hadoop) - Vice President

    Pune, Maharashtra Citigroup

    Posted 15 days ago

    Job Viewed

    Tap Again To Close

    Job Description

    The Applications Development Technology Lead Analyst is a senior level position responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to lead applications systems analysis and programming activities.
    **Responsibilities:**
    + Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
    + Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
    + Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
    + Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
    + Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
    + Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
    + Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
    + Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
    **Qualifications:**
    + **10+ years of relevant experience in Apps Development or systems analysis role using Java, Spark, HDFS, Hive, Hadoop**
    + **i)Hands-on development expertise in Java with Spark.** **Extensive knowledge of HDFS, Hive(Relevant experience of around4-5 years .** **ii) Hands on knowledge of core Java concepts and framework such as Spring Boot, Microservices and well versed with OOPs concepts and design patterns.** **iii) Familiarity with data formats like Avro, Parquet, CSV, JSON.** **iv) Java knowledge with advanced skills in multithreading and multiprocessing, along with extensive experience in efficiently processing large-scale data.**
    + Extensive experience system analysis and in programming of software applications.
    + Experience in managing and implementing successful projects.
    + Subject Matter Expert (SME) in at least one area of Applications Development.
    + Ability to adjust priorities quickly as circumstances dictate.
    + Demonstrated leadership and project management skills.
    + Consistently demonstrates clear and concise written and verbal communication.
    **Skills required:**
    + **Highly experienced and skilled Java technical lead with 10-12 years of experience with software building and platform engineering.**
    + **Extensive development expertise in building the high scaled and performant software platforms for data computation and processing.**
    + **Expert level knowledge of core Java concepts and framework such as Spring Boot, Microservices and well versed with OOPs concepts and design patterns.**
    + **Java expert with advanced skills in multithreading and multiprocessing, along with extensive experience in efficiently processing large-scale data.**
    + **Expertise and hands-on experience on working with Apache Spark using Java and understanding of the Bigdata ecosystem and design principles**
    + **Hands-on experience on Unix and python/shell scripting.**
    + **Good knowledge in Hadoop, YARN, Hive, Spark, and Spark SQL with extensive high volume of data processing pipeline development.**
    + **Strong computer science fundamentals in data structures, algorithms, databases, and operating systems.**
    + **Highly experienced with Unix based operating systems and shell scripting.**
    + **Strong analytical and logical skills.**
    + **Hands-on experience in writing SQL queries.**
    + **Experience with source code management tools such as Bitbucket, Git etc.**
    + **Experience working with banking domain like pricing, risk etc. is plus.**
    + **Consistently demonstrates clear and concise written and verbal communication**
    **Education:**
    + Bachelor's degree/University degree or equivalent experience
    + Master's degree preferred
    This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
    ---
    **Job Family Group:**
    Technology
    ---
    **Job Family:**
    Applications Development
    ---
    **Time Type:**
    Full time
    ---
    **Most Relevant Skills**
    Please see the requirements listed above.
    ---
    **Other Relevant Skills**
    For complementary skills, please see above and/or contact the recruiter.
    ---
    _Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
    _If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
    _View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
    Citi is an equal opportunity and affirmative action employer.
    Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.
    This advertiser has chosen not to accept applicants from your region.
    Be The First To Know

    About the latest Hdfs Jobs in India !

    Nearby Locations

    Other Jobs Near Me

    Industry

    1. request_quote Accounting
    2. work Administrative
    3. eco Agriculture Forestry
    4. smart_toy AI & Emerging Technologies
    5. school Apprenticeships & Trainee
    6. apartment Architecture
    7. palette Arts & Entertainment
    8. directions_car Automotive
    9. flight_takeoff Aviation
    10. account_balance Banking & Finance
    11. local_florist Beauty & Wellness
    12. restaurant Catering
    13. volunteer_activism Charity & Voluntary
    14. science Chemical Engineering
    15. child_friendly Childcare
    16. foundation Civil Engineering
    17. clean_hands Cleaning & Sanitation
    18. diversity_3 Community & Social Care
    19. construction Construction
    20. brush Creative & Digital
    21. currency_bitcoin Crypto & Blockchain
    22. support_agent Customer Service & Helpdesk
    23. medical_services Dental
    24. medical_services Driving & Transport
    25. medical_services E Commerce & Social Media
    26. school Education & Teaching
    27. electrical_services Electrical Engineering
    28. bolt Energy
    29. local_mall Fmcg
    30. gavel Government & Non Profit
    31. emoji_events Graduate
    32. health_and_safety Healthcare
    33. beach_access Hospitality & Tourism
    34. groups Human Resources
    35. precision_manufacturing Industrial Engineering
    36. security Information Security
    37. handyman Installation & Maintenance
    38. policy Insurance
    39. code IT & Software
    40. gavel Legal
    41. sports_soccer Leisure & Sports
    42. inventory_2 Logistics & Warehousing
    43. supervisor_account Management
    44. supervisor_account Management Consultancy
    45. supervisor_account Manufacturing & Production
    46. campaign Marketing
    47. build Mechanical Engineering
    48. perm_media Media & PR
    49. local_hospital Medical
    50. local_hospital Military & Public Safety
    51. local_hospital Mining
    52. medical_services Nursing
    53. local_gas_station Oil & Gas
    54. biotech Pharmaceutical
    55. checklist_rtl Project Management
    56. shopping_bag Purchasing
    57. home_work Real Estate
    58. person_search Recruitment Consultancy
    59. store Retail
    60. point_of_sale Sales
    61. science Scientific Research & Development
    62. wifi Telecoms
    63. psychology Therapy
    64. pets Veterinary
    View All Hdfs Jobs