amazon neptune case study

Siemens use cases with metaphactory and Amazon Neptune presented at AWS re:Invent 2017

CASE STUDY - METAPHACTORY & AMAZON NEPTUNE

Siemens and metaphacts were among the first Amazon Neptune users and showed several use cases for enterprise knowledge graphs using the new service and metaphactory .

In a joint presentation on Dec 1, 2017 at AWS re:Invent , Siemens and metaphacts provided details on how they use Amazon Neptune, standalone graph stores, and the metaphactory platform to drive use cases for enterprise knowledge graphs. Leveraging this combined solution, Siemens powers an intelligent knowledge hub combining structured and unstructured data from different sources and business units in a unified data space. This knowledge hub can also be used as a smart knowledge factory generating new knowledge.

"We are looking forward to investigate Amazon Neptune in depth and we are glad to have a partner in metaphacts which understands our data management needs and has supported us in driving knowledge graph use cases for our business and for our customers", says Thomas Hubauer, Portfolio Project Manager for Knowledge Graph & Semantics at Siemens Corporate Technology.

Building digital twin

Gas turbine maintenance

Building automation

Factory monitoring

Press Release

Siemens and metaphacts showcase Enterprise Knowledge Graphs on Amazon Neptune »

Recorded session from AWS re:Invent 2017 »

Amazon Neptune: The Ultimate Guide to AWS’s Graph Database Service

Amazon Neptune stands out as a pivotal service within AWS’s extensive portfolio, offering a fully managed graph database engineered to manage highly connected datasets efficiently. This guide delves into the core aspects of Amazon Neptune, highlighting its features, use cases, and the value it brings to modern applications requiring intricate data relationships.

Table of Contents

Understanding amazon neptune, what is amazon neptune.

Amazon Neptune is a fast, reliable, and fully managed graph database service provided by Amazon Web Services (AWS) . It is specifically designed for storing and querying highly connected data, making it an ideal choice for applications that rely on complex datasets such as social networks, recommendation engines, and fraud detection systems. Neptune supports popular graph models like Property Graph and W3C’s RDF, along with their respective query languages, Apache TinkerPop Gremlin and SPARQL.

What is a Graph Database?

A graph, in the context of database systems, is a structure consisting of nodes (also known as vertices) and edges (links or relationships). Each node represents an entity, such as a person, place, thing, or concept, while edges depict the relationships or interactions between these entities. This flexible and intuitive model mirrors the interconnected nature of data in real-world applications, from social networks where users are connected by friendships to business ecosystems where transactions link customers, products, and vendors. Graphs excel in scenarios where relationships are as critical as the data points themselves, enabling queries that explore the depth and breadth of connections to uncover patterns, insights, and opportunities hidden within the data. By prioritizing relationships, graph databases like Amazon Neptune facilitate a deeper analysis of data relationships, making them indispensable tools for applications that rely on the intricate interplay between diverse data elements.

How Amazon Neptune Works

Amazon Neptune is designed to offer a robust, scalable solution for managing graph data. At its core, Neptune stores and processes data in formats that are optimized for high connectivity and complex relationship traversals, enabling rapid query responses and data retrieval. The service employs a distributed, fault-tolerant architecture that ensures data availability and resilience, automatically replicating data across multiple Availability Zones in an AWS Region. This architecture not only facilitates high availability but also enables Neptune to provide consistent performance at scale, regardless of the size of the dataset or the complexity of the queries. By leveraging fast, parallel bulk loading from Amazon S3 , Neptune allows for efficient data ingestion, making it easier to get started with large datasets. Additionally, Neptune’s query processing engine is designed to optimize query execution, automatically translating high-level graph queries into efficient, low-level operations. This sophisticated processing capability, combined with Neptune’s support for both Gremlin and SPARQL query languages, empowers developers to build flexible, powerful applications that can navigate complex data relationships with ease.

Key Features and Benefits of Amazon Neptune

High performance and scalability.

Neptune is engineered from the ground up for high performance and is capable of handling over 100,000 graph queries per second. It supports automatic scaling, allowing for up to 15 read replicas across three Availability Zones to extend read capacity and maintain low-latency access to data.

Fully Managed Service

As a fully managed service, Neptune abstracts away the complexities of hardware provisioning, software patching, and setup, enabling developers to focus on building their applications. AWS takes care of the operational burden, offering a seamless experience for managing graph databases.

Security and Compliance

Hosted within Amazon Virtual Private Cloud (VPC), Neptune ensures data isolation and secure connections. It integrates with AWS Identity and Access Management (IAM) for access control and supports encryption at rest and in transit, meeting the stringent security and compliance requirements of various industries.

Seamless Integration with AWS Ecosystem

Neptune is designed to work seamlessly with the broader AWS ecosystem, including services like Amazon S3 for data storage, AWS Lambda for executing code in response to triggers, and Amazon Kinesis for real-time data streaming. This integration enables developers to build comprehensive, cloud-native applications that leverage the strengths of AWS services.

Serverless and Global Database Capabilities

Neptune offers a serverless option, eliminating the need to manage database capacity and allowing you to pay only for the resources your application consumes. Additionally, the Neptune Global Database feature enables the deployment of a single graph database across multiple AWS Regions , reducing latency and improving disaster recovery capabilities.

Machine Learning Integration

With Neptune ML, developers can enhance their applications with machine learning capabilities. Powered by Amazon SageMaker , Neptune ML automates the heavy lifting involved in selecting, training, and optimizing machine learning models, making it easier to generate predictions directly from graph data.

Multi-Model Support

Neptune supports multiple graph models, including Property Graph and RDF (Resource Description Framework), along with their respective query languages, Apache TinkerPop Gremlin and SPARQL. This multi-model support allows developers to choose the most appropriate data model for their specific application needs, whether they’re working with highly connected data sets or leveraging semantic web data.

Continuous Backup and Point-in-Time Recovery

Neptune provides continuous backup to Amazon S3, enabling point-in-time recovery for your databases. This feature ensures data durability and recoverability, allowing you to restore your database to any second within the backup retention period, which can be crucial for maintaining data integrity in the event of accidental deletions or database corruption.

These key features make Amazon Neptune a powerful and versatile choice for developers and organizations looking to harness the power of graph databases to manage complex and highly connected data sets efficiently.

Use Cases for Amazon Neptune

Amazon Neptune’s versatility shines across various domains, demonstrating its capability to handle complex, highly connected datasets with ease. Here’s an expanded look at its use cases, incorporating insights from the additional resources.

Building Identity Graphs

Identity graphs play a crucial role in understanding customer behavior across multiple platforms and devices. Amazon Neptune facilitates the creation of comprehensive identity graphs by linking customer identifiers such as devices, email addresses, and social media profiles. This unified view enables personalized marketing strategies and targeted advertising, enhancing customer engagement and conversion rates. By leveraging Neptune, organizations can achieve a 360° view of their customers, understanding their journey and preferences in unprecedented detail. This capability is particularly beneficial for advertising technology companies, marketing agencies, and businesses aiming to enhance customer data platforms with a focus on privacy regulation compliance.

Enhancing Fraud Detection Mechanisms

Fraud detection is a critical concern for businesses worldwide, with organizations losing billions annually to fraudulent activities. Neptune’s graph database structure is ideally suited for uncovering complex fraud schemes, including those perpetrated by sophisticated fraud rings. By visualizing the relationships between transactions, accounts, and other entities, Neptune enables real-time detection of fraudulent patterns. This approach is not only more efficient but also more effective than traditional methods, allowing for the proactive prevention of fraud across various sectors, including banking, insurance, and e-commerce.

Leveraging Machine Learning for Graph Data

Neptune ML revolutionizes the way predictions are made using graph data. By employing graph neural networks (GNNs), Neptune ML significantly enhances the accuracy of predictions, surpassing traditional machine learning methods that struggle with the complex relationships inherent in graph data. This capability is crucial for applications ranging from recommendation systems to fraud detection, where understanding the nuanced connections between data points can lead to more informed and accurate outcomes. Neptune ML democratizes access to advanced machine learning techniques, enabling developers to deploy sophisticated models without deep expertise in data science.

Securing IT Infrastructure with Security Graphs

Security graphs represent a novel approach to managing and enhancing the security of IT infrastructure. By mapping out the relationships between assets, users, and access permissions, Neptune helps organizations identify vulnerabilities and potential attack vectors within their systems. This comprehensive view is invaluable for proactive threat detection, incident response, and compliance with security policies. Security graphs are particularly effective in layered security environments, where understanding the interplay between different security measures can reveal gaps and redundancies. With Neptune, organizations can model their security posture as a graph, enabling more effective defense strategies against both internal and external threats.

Getting Started with Amazon Neptune

Creating and managing a neptune database.

Setting up a Neptune database involves a few straightforward steps, starting with the AWS Management Console . Users can configure their database instances, including security settings and network access, and begin loading data to start building their graph applications.

Querying Data with Gremlin and SPARQL

Neptune supports two powerful query languages: Gremlin for Property Graphs and SPARQL for RDF models. These languages enable developers to efficiently query and manipulate highly connected data, uncovering insights that would be challenging to obtain with traditional databases.

Integrating Neptune with AWS Services

Neptune’s value is further amplified when integrated with other AWS services, such as Amazon S3 for data storage, AWS Lambda for serverless computing, and Amazon SageMaker for machine learning. These integrations allow for building sophisticated, scalable applications that leverage the full power of AWS’s cloud ecosystem.

Best Practices for Using Amazon Neptune

Adhering to best practices is crucial to maximize the benefits of Amazon Neptune in managing highly connected datasets. These guidelines not only ensure optimal performance but also enhance the security, scalability, and maintainability of your graph database applications.

Optimize Data Modeling

Data modeling is a critical step in the effective use of Neptune. Design your graph model to reflect the queries you anticipate running. This involves structuring nodes, edges, and properties in a way that aligns with your application’s access patterns. For Property Graphs, use labels and property keys efficiently to facilitate fast retrieval. For RDF graphs, leverage named graphs and efficient IRI (Internationalized Resource Identifiers) strategies to categorize and access your data effectively.

Utilize Indexing Strategically

While Neptune automatically manages indexing, understanding how your queries interact with these indexes can lead to performance improvements. For Gremlin, ensure that your queries are using property-based filtering early in the traversal to take advantage of Neptune’s indexing. In SPARQL queries, use FILTER clauses judiciously and consider graph-specific optimizations for faster query execution.

Manage Connections Wisely

Connection management is pivotal in ensuring the high availability and responsiveness of your Neptune database. Implement connection pooling to reduce the overhead of establishing connections to Neptune. This is particularly important for applications with high request rates. Additionally, monitor your connection usage and adjust your pool size based on the workload to avoid throttling.

Scale Effectively

Leverage Neptune’s scalability features to accommodate your application’s growth. Use read replicas to distribute query load and enhance read throughput, especially for read-intensive applications. Monitor your database’s performance metrics through Amazon CloudWatch to identify when scaling is needed, whether it’s adding more replicas or resizing your instances.

Ensure Data Security

Data security in Neptune involves multiple layers, including network security, access control, and data encryption. Use Amazon VPC to isolate your database and control access with security groups. Implement fine-grained access control with IAM policies and database authentication. Enable encryption at rest and in transit to protect your data from unauthorized access.

Backup and Recovery

Regularly back up your Neptune database to safeguard against data loss. Neptune’s continuous backup feature allows for point-in-time recovery, enabling you to restore your database to any second within the backup retention period. Test your backup and recovery procedures to ensure they meet your business continuity requirements.

Monitor and Audit

Continuous monitoring and auditing are essential for maintaining the health and security of your Neptune database. Use Amazon CloudWatch to track operational metrics and set alarms for anomalous activity. Enable logging and use AWS CloudTrail for auditing access and changes to your Neptune environment, helping you comply with regulatory requirements and internal policies.

Leverage Neptune Features and AWS Ecosystem

Take full advantage of Neptune’s features, such as the Gremlin and SPARQL query languages, to efficiently interact with your graph data. Integrate Neptune with other AWS services like Amazon SageMaker for machine learning, AWS Lambda for serverless computing, and Amazon Elasticsearch Service for advanced search capabilities to build comprehensive, cloud-native applications.

By following these best practices, you can optimize your use of Amazon Neptune, ensuring that your graph database applications are secure, scalable, and performant. This will enable you to derive maximum value from your highly connected datasets, driving insights and innovation in your organization.

Amazon Neptune represents a significant advancement in managing highly connected data, offering unparalleled performance, scalability, and ease of use. Whether for driving recommendation engines, enhancing fraud detection, or building comprehensive knowledge graphs, Neptune provides a robust, secure, and fully managed graph database solution. As businesses continue to navigate the complexities of modern data landscapes, Neptune stands ready to support their most challenging graph database needs.

Additional Resources

• Amazon Neptune Pricing (Understand the cost structure of Amazon Neptune and how pricing works for different configurations and usage patterns.)
• Amazon Neptune FAQs (Find answers to frequently asked questions about Amazon Neptune, covering a wide range of topics from features to management.)
• AWS Learning Path: Getting Started with Amazon Neptune (A learning resource for beginners to get started with Amazon Neptune, including tutorials and best practices.)
• Bulk Load Data to Neptune – Gremlin Format (Learn how to prepare and bulk load data into Neptune using the Gremlin format for graph data.)
• Accessing Neptune Graphs using Gremlin (A guide on how to access and query your graph data in Neptune using the Gremlin query language.)
• Bulk Load Data to Neptune – openCypher Format (Instructions on how to bulk load data into Neptune using the openCypher format, suitable for property graph models.)
• Accessing Neptune Graphs using openCypher (Explore how to use the openCypher query language to access and interact with your graph data in Neptune.)
• Bulk Load Data to Neptune – RDF Format (Discover how to bulk load RDF formatted data into Neptune, enabling efficient data ingestion for RDF graph models.)
• Accessing Neptune Graphs using SPARQL (A comprehensive guide to querying RDF graph data in Neptune using the SPARQL query language.)

These resources provide valuable information and practical guidance for anyone looking to leverage Amazon Neptune for graph database solutions, from beginners to advanced users.

Ready to get started with Amazon Neptune?

Other aws guides.

AWS Outposts: Enhancing On-Premises and Cloud Integration

AWS IoT Core: Key Features and Pricing Explained

Optimizing Software Development: The Power of Amazon CodeGuru

AWS Transit Gateway: Streamlining Complex Network Architectures

AWS X-Ray for Application Insight and Debugging

Text Insights with AWS Comprehend: A Comprehensive Guide

Get the latest articles and news about AWS

I have read and agree with Cloudvisor's Privacy Policy .

Take advantage of instant discounts on your AWS and Cloudfront services

AWS Cost Optimization

Squeeze the best performance out of your AWS infrastructure for less money

Well-Architected Framework Review

Ensure you're following AWS best practices with a free annual WAFR review

Monitoring Service

24/7 monitoring catches any potential issues before they turn into a problem

Data Engineering Services

Make the most of your data with optimization, analysis, and automation

Migration to AWS

Seamlessly transfer your cloud infrastructure to AWS with minimal downtime

AWS Security

Protect your AWS infrastructure with sophisticated security tools and consultation

AWS Marketplace

Access the best tools for your use case via the AWS Marketplace

• For Startups
• Case Studies

Dive into our latest insights, trends, and tips on cloud technology.

Your comprehensive resource for mastering AWS services.

Join our interactive webinars to learn from cloud experts.

Whitepapers

Explore in-depth analyses and research on cloud strategies.

• Free consultation

Amazon Neptune

Amazon Neptune lets you build interactive graph applications that can query billions of connections in milliseconds. Complexity and difficulty in tuning SQL queries for heavily connected data are two of the most common drawbacks of this technology. Apache TinkerPop Gremlin and W3C’s SPARQL are two prominent graph query languages that you may use with Amazon Neptune instead to execute powerful searches on related data.

Using Amazon Neptune , you are able to construct interactive graph applications to query billions of connections in a matter of milliseconds. The most typical disadvantages of utilising this technology include its complexity as well as the difficulty in fine-tuning SQL searches for data that is densely related. You may run powerful searches on related data by using Amazon Neptune and one of the main graph query languages, such as Apache TinkerPop Gremlin or W3C’s SPARQL. Both of these languages are available from the W3C. When you accomplish this, your code will become simpler, and the creation of relationship-processing apps will move forward more quickly. It is anticipated that Amazon Neptune would achieve availability rates greater than 99.99 percent by combining the database engine with an SSD-backed virtualized storage layer that has been adapted specifically for database workloads. The self-healing Neptune storage is fault tolerant, and it is possible to fix disc faults in the background without having an effect on the availability of the database. The automatic identification and restart of Neptune’s databases are designed to eliminate the need for crash recovery or a full database cache rebuild in the event that either of these processes become necessary. In the event that the entire instance crashes, Neptune is designed to automatically fail over to one of up to 15 read replicas. Launching an Amazon Neptune database instance is a simple process that can be accomplished in a matter of seconds using the Neptune Management Console. Neptune automatically scales storage to provide constant performance by increasing storage capacity and rebalancing input/output operations.

Amazon Neptune is not built on a relational database. Instead, it is a high-performance graph database engine that has been specifically designed for Amazon’s platform. Neptune is optimized for storing and querying complex graph data efficiently. It utilizes a scale-up architecture that is optimized for in-memory operations, allowing it to provide quick query evaluation capabilities even for large graphs.

How it Works

Effortless and Expandable

• Queries on graphs can benefit from high throughput and low latency.

It was built exclusively for Amazon and is a high-performance graph database engine To quickly evaluate queries over large graphs, Neptune uses a scale-up, in-memory optimised architecture, which effectively stores and navigates graph data. Neptune can be used with Gremlin or SPARQL to conduct fast and simple queries.

• Database Compute Resources can be easily scaled.

When scaling up or down your production cluster, you may do it with only a few clicks in the AWS Management Console. In most cases, scaling activities are completed within a few seconds.

• Automated Storage Scalability

Amazon Neptune will automatically raise the volume of your database as your storage needs grow. You have the option of using up to 64 terabytes of storage space. There’s no need to add more database storage to keep up with future expansion.

In addition to the automatic adjustment of database volume, it is worth noting that the minimum storage capacity for an Amazon Neptune database is 10GB. This ensures that you have a solid starting point for your storage needs. From there, as your usage increases, the database storage will seamlessly scale up in 10GB increments. This scalability allows your Amazon Neptune storage to grow organically without any impact on database performance.

With a maximum storage limit of 64TB, you can rest assured that your database can handle extensive data storage requirements. The incremental growth in storage capacity is designed to accommodate your evolving needs, eliminating the need for proactive planning or adding more storage space manually.

• Read Replicas with Low Latency.

Increase read throughput by creating up to 15 database read replicas for high-volume application requests. Amazon Neptune replicas offer cost savings and the avoidance of writing to replica nodes. The latency time for read requests can be reduced to single-digit milliseconds on numerous occasions by using more processing power. Applications don’t have to worry about keeping track of new and removed copies because Neptune only has one endpoint for read queries.

Download list of all AWS Services PDF

Download our free PDF list of all AWS services. In this list, you will get all of the AWS services in a PDF file that contains  descriptions and links on how to get started.

Quality and Reliability

• Incident Management as well as Repair.

Your Amazon Neptune database and underlying EC2 instance are always in good shape, so you can relax. The instance that runs your database is responsible for restarting it and any associated processes. Because the database redo logs do not need to be replayed, instance restart times with Neptune recovery are often under 30 seconds. It also shields the database buffer cache from the processes of the database, allowing it to withstand a database restart.

• Using Read Replicas in Multi-AZ Deployments

In the event of an instance failure, Neptune will immediately transition to one of up to 15 Neptune replicas located in any of three Availability Zones. If there are no Neptune replicas, Neptune will attempt to automatically construct a new database instance in the event of a failure.

• Self-healing and fault-tolerant Storage

Replication occurs six times across three availability zones for every 10GB of database storage. Amazon Neptune’s fault-tolerant storage can lose up to two copies of data without affecting database read availability in order to ensure database write availability. Neptune’s storage is self-healing for all of its data blocks and discs.

• Point-in-time recovery and background incremental backups.

To recover your instance, you can use Amazon Neptune’s backup tool. You can use this method to recover your database up to the last five minutes of its retention period. You can specify a retention time for your automated backups of up to 35 days. Amazon S3, a service meant to guarantee 99.999999999 percent uptime, is used to store automated backups. The Neptune backups have no effect on database performance.

• Snapshots of the database

In Amazon S3, database snapshots are replicas of your instance that are created on demand and kept until they are permanently removed. For time and space savings, they use automated incremental snapshots. You can utilise a Database Snapshot at any time to create a new instance of the database.

Need help on AWS?

AWS Partners, such as AllCode, are trusted and recommended by Amazon Web Services to help you deliver with confidence. AllCode employs the same mission-critical best practices and services that power Amazon’s monstrous ecommerce platform.

APIs for the Open Graph

• Gremlin supports Apache TinkerPop’s Property Graph

Neptune offers robust support for querying with two powerful languages: Gremlin for Property Graphs and SPARQL for RDF models. Property graphs have recently gained popularity due to their familiarity with relational models, and Gremlin traversal language simplifies the exploration of these graphs. Amazon Neptune backs the Property Graph concept and provides a Gremlin Websockets server compatible with the latest TinkerPop version. Additionally, Neptune, an Amazon service, boasts comprehensive backup and restore functionality for databases, allowing users to easily build Gremlin traversals over property graphs by modifying the Gremlin service configuration for seamless integration into existing applications.

When managing highly connected datasets using Amazon Neptune, it is essential to consider a range of best practices to ensure optimal performance, security, and scalability. Data modeling plays a crucial role in the effective utilization of Neptune. By designing your graph model to align with anticipated queries and structuring nodes, edges, and properties efficiently, you can enhance the accessibility and retrieval speed of your data. Additionally, understanding and utilizing indexing strategically can lead to significant performance improvements. Whether it’s Gremlin queries or SPARQL queries, optimizing index usage can result in faster query execution and improved overall efficiency.

To ensure the security of your databases, Neptune allows you to produce and manage encryption keys using the AWS Key Management Service (KMS). By encrypting your Neptune databases, you can safeguard your data stored in the underlying storage and automated backups, snapshots, or replicas within the same cluster.

Regarding data security, Neptune offers robust security features, including network isolation with Amazon VPC and encrypted IPsec VPNs. You can safeguard your data from unauthorized access by implementing fine-grained access control with IAM policies and database authentication. Regularly backing up your Neptune database and testing recovery procedures are essential steps to protect against data loss and ensure business continuity.

Stay informed about critical database events through email and SMS alerts. Neptune allows subscribing to database events, such as automated failovers, through the intuitive AWS Management Console. This proactive approach keeps you updated and enables prompt actions in response to any database issues.

• W3C’s RDF 1.1 and SPARQL specifications are supported.

Neptune extends its support to W3C’s RDF 1.1 and SPARQL specifications. RDF is widely accepted in complex information domains, and Neptune enables users to access RDF datasets like Wikidata and PubChem through its SPARQL endpoint. Leveraging an HTTP REST interface, developers can effortlessly integrate Neptune’s RDF and SPARQL support into both new and existing graph applications.

Exceptionally Secure

• Isolation of the Network

Security in AWS Neptune is of paramount importance, and Amazon Web Services (AWS) offers a range of robust security features to protect your data. One key aspect of security is the isolation of the network, which can be achieved by utilizing Amazon Virtual Private Cloud (VPC) and encrypted IPsec VPNs. This allows you to connect to Amazon Neptune within your own virtual network, ensuring that your database remains segregated from other resources.

Another crucial security measure is managing permissions at the resource level. With Amazon Neptune, you have the ability to govern access to specific resources such as database snapshots and parameter groups. This enables you to control the actions that AWS Identity and Access Management (IAM) users and groups can perform on these resources. By tagging your Neptune resources, you can further refine permissions based on specific criteria. For instance, you can restrict changes or deletions of “Production” database instances to only Database Administrators, while allowing developers to modify “Development” instances, all through well-defined IAM rules.

Encryption is a vital component of database security, and Amazon Neptune provides robust encryption features. You can generate and manage encryption keys through AWS Key Management Service (KMS) to safeguard your databases. This ensures that not only the data stored in the underlying storage, but also automated backups, snapshots, and replicas within the same cluster, are protected with Neptune-encrypted database instances.

• Permissions at the Resource Level

This dynamic partnership enables Neptune to leverage certain management features that ensure optimal performance and security. For instance, Neptune benefits from Amazon RDS’s expertise in instance lifecycle management, allowing seamless management and scaling of Neptune instances. Additionally, Neptune leverages encryption-at-rest capabilities with Amazon Key Management Service (KMS) keys, providing robust data protection. Moreover, Neptune takes advantage of Amazon RDS’s security groups management, ensuring that access to Neptune instances is precisely controlled and secured.

Just remember Amazon RDS permissions and resources are required to use Amazon Neptune because Neptune leverages operational technology from Amazon RDS, including instance lifecycle management, encryption-at-rest using Amazon Key Management Service (KMS) keys, and security groups management. By utilizing the capabilities of Amazon RDS, Neptune can provide high-performance graph database services that are specifically designed for use on the Amazon platform.

It is possible to produce and manage the encryption keys used by Amazon Neptune to safeguard your databases using AWS Key Management Service (KMS). To ensure that any data stored in the underlying storage as well as automated backups, snapshots or replicas in the same cluster is safeguarded by Neptune-encrypted database instances.

Recording database events with Amazon Neptune has minimal impact on database performance. Studying logs now can help with database management, security, governance, and regulatory compliance issues in the future In addition, you can transmit audit logs to the Amazon CloudWatch service to keep track of what’s happening in your environment.

Completely Organized

• Simple to Use

It’s easy to use Amazon Neptune. Create a new Neptune database instance using the AWS Management Console. The database instance class you supplied is pre-configured in Neptune database instances. You may create a database and attach it to your app in just a few minutes. It is possible to fine-tune your database by using Database Parameter Groups.

• Operate easily

Amazon Neptune is a powerful and user-friendly graph database solution. It simplifies the process of building and managing high-performance graph databases. With Neptune’s API, you can create graph databases without the need for specific graph indexes, making it easier to work with your data. The API also provides a timeout and memory limit feature, ensuring that queries do not consume excessive memory or time out. To monitor the performance of your Neptune database instances, Amazon CloudWatch is used. This allows you to keep track of essential operational indicators such as CPU utilization, RAM usage, storage, query performance, and active connections. The AWS Management Console provides a convenient interface where you can access and analyze these metrics for your database instances running on Amazon Web Services.

Alternatively, users can quickly clone multi-terabyte database clusters. This cloning capability is beneficial for various purposes, including application development, testing, database upgrades, and analytical queries. By having immediate access to cloned data, you can efficiently perform software development tasks, updates, and analytics. The process of cloning an Amazon Neptune database is straightforward and can be done with just a few clicks in the Management Console. The clone is replicated across three Availability Zones, ensuring redundancy and high availability.

• Measuring and metric

Amazon Neptune uses Amazon CloudWatch to keep tabs on your database instances. The AWS Management Console displays over 20 critical database operational indicators, such as CPU, RAM, storage, query performance, and active connections, for database instances running on Amazon Web Services’s cloud platform.

• Auto-Patching

In order to maintain your database patched, use Amazon Neptune. It is possible to control patching with Database Engine Version Management (DEV).

• Database Event Alerts

Email and SMS alerts can be used to notify users of critical database events, such as an automated failover. Subscribing to database events can be done through AWS Management Console.

• Database Cloning

Neptune from Amazon allows for multi-terabyte database clusters to be quickly cloned in minutes. Application development, testing, database upgrades, and analytical queries can all benefit from cloning. Having immediate access to data improves software development and updates, as well as analytics.

You can clone an Amazon Neptune database with a few mouse clicks in the Management Console. Three Availability Zones are used to reproduce the clone.

Rapid Bulk Data Loading

• Bagging of Property Graphs

Amazon Neptune’s bulk loading capability makes it easy to load large amounts of data from S3. A REST interface is used to do this. CSV delimiter format is used to load data into the nodes and edges of the graph. Neptune Property Graph bulk loading has more information.

• RDF Bulk Load

It is possible to load RDF data stored in S3 using Amazon Neptune in a fast and efficient manner. A REST interface is used to do this. N-Triples (NT), N-Quads (NQ), RDF/XML, and Turtle RDF are all supported serialisations. Neptune RDF bulk loading documentation can be found here.

General Use Cases

1. Social Networking Neptune can be utilized to build social networking applications by efficiently processing large sets of user-profiles and interactions. It enables prioritizing the display of recent updates from family, friends, and user-liked content, enhancing the user experience.

2. Recommendation Engines With Neptune’s graph database capabilities, relationships between different information can be stored and queried swiftly. This functionality facilitates personalized and relevant recommendations, such as suggesting products to users based on their interests, friends’ preferences, and purchase history.

3. Fraud Detection Neptune’s real-time processing capabilities make it an excellent choice for fraud detection applications. By analyzing financial and purchase transactions, Neptune can identify patterns indicative of fraud, such as multiple people sharing the same IP address but residing at different physical addresses or multiple individuals linked to a single email address.

4. Knowledge Graphs Neptune allows the creation of knowledge graphs, which store information in a graph model and enable graph-based queries. This feature proves valuable for helping users navigate highly interconnected datasets. For example, in an art-related knowledge graph, users interested in one artwork can discover related works by the same artist or explore other pieces in the same museum.

5. Life Sciences In the life sciences field, Neptune provides a secure and efficient means of storing and querying information. It supports encryption at rest for sensitive data and can be used to model and search for relationships within biological and genetic data. For instance, Neptune can help identify potential gene-disease associations by analyzing protein pathways.

6. Network/IT Operations Neptune can be applied to network and IT operations for rapid analysis and understanding of anomalous events. By querying for graph patterns using event attributes, Neptune enables efficient identification and tracing of the root cause of issues. For example, if a malicious file is detected on a host, Neptune can assist in tracking its origin and propagation within the network.

Cost-Effectiveness

Pay Only for the Services You Utilize

Amazon Neptune pricing is designed to offer cost-effective solutions for your database needs. With Neptune, you eliminate the need for a substantial upfront expenditure, paying an hourly charge for each instance you start. This means you can create and stop instances as needed without incurring unnecessary costs. Additionally, you only pay for the storage you utilize, making it a cost-efficient option for managing your data.

When it comes to billing, Amazon Neptune ensures transparency and flexibility. No long-term commitments or upfront payments are required, allowing you to pay per hour for On-Demand instances. Discounts are available on read-write primary instances and Amazon Neptune replicas, which are utilized to boost reads and enhance failover. In terms of performance, Amazon Neptune stands out with its support for 15 read replicas, allowing for increased scalability and fault tolerance. Additionally, the database can handle 100,000s of queries per second, ensuring high throughput for demanding workloads. Neptune database storage charges are billed in GB-month increments, while I/O charges are billed in million-request increments. Remember, apart from the storage and I/Os you consume, there is no need to allocate resources in advance when using Neptune.

Storage charges are billed in GB-month increments, and I/O charges are billed in million-request increments, ensuring that you are only charged for the resources you use. Any database cluster snapshots and automated database backups customers request incur a per GB-month charge for backup storage. Data transfer charges are based on the volume of data sent in and out of the planet, providing a clear understanding of your data transfer costs.

In the Ready State, the Amazon Neptune Workbench charges by the instance hour and allows you to interact with your Neptune cluster using Jupyter notebooks hosted by Amazon SageMaker. With Amazon Neptune, you have the flexibility to scale your resources based on your needs, ensuring cost-effectiveness and efficiency in managing your database workloads

Curious About AWS Pricing?

Pricing Amazon Web Services may seem tricky, but the AWS Pricing Calculator makes it quick and easy to get an estimate!

• Documentation
• Partner Network
• AWS Marketplace
• Explore More
• Bahasa Indonesia
• AWS Management Console
• Account Settings
• Billing & Cost Management
• Security Credentials
• AWS Personal Health Dashboard

Getting Started with Amazon Neptune

Databases | 7 Videos | About 9 Hours

Learning Path

Access resources.

Download slide decks, check out additional helpful resources, and receive learning path email notifications.

Table of Contents

1. Create and Connect to Amazon Neptune Database

• Options to create a database, and configure security
• How to connect to Neptune using workbench and API
• Bulk load data into the cluster for querying

2. Data Modeling for Amazon Neptune

• The basics of the RDF and property graph data models
• How to apply these models when designing an Amazon Neptune graph application
• How to evolve your application graph data model to accommodate new use cases and queries

3. Writing Your Graph Queries Against Amazon Neptune

• The basics of Gremlin and SPARQL query syntax to insert, query and delete data
• Advanced query concepts such as federated query, traversal strategies
• Best practices writing Gremlin and SPARQL queries

4. Building End-to-End Graph Use Cases with Amazon Neptune

• How Neptune can be used to build an identity graph
• How Neptune can be used to build knowledge graph
• How Neptune can be used in fraud detection solution

5. Managing Your Amazon Neptune Database

• Monitor your Neptune cluster using CloudWatch
• How to right size your Neptune cluster
• Info about snapshots, backups, and software upgrades

6. Troubleshooting Amazon Neptune - Query Performance

• Graph data model and indexes used by Neptune
• Workload performance benchmarking and profiling a Gremlin or SPARQL query
• Troubleshooting query performance

7. Integrating Amazon Neptune with Other AWS Services

• How to integrate Amazon Neptune with other AWS services using Neptune Streams
• How to use AppSync with Amazon Neptune
• How to use AWS Glue with Amazon Neptune

Description

Amazon Neptune is a fast, reliable, fully managed graph database service that makes it easy to build and run applications that work with highly connected data sets. The core of Amazon Neptune is a purpose-built, high-performance graph database engine optimized for storing billions of relationships and querying the graph with milliseconds latency. You can build knowledge graphs, fraud graphs, identity graphs, recommendation engines, master data management, and network security applications using Neptune. In this video series you will learn how to get started with Amazon Neptune. You will learn the use cases and the basics of Neptune including creating and managing your cluster, building popular graph models Property Graph and W3C’s RDF, writing queries using Apache TinkerPop Gremlin and SPARQL, troubleshooting performance, and integrating with tools and services such as Elasticsearch and AWS Glue.

Learning Objectives

Objective One

Learn how to build your graph applications by creating and managing your cluster

Objective Two

Build your Property Graph model or RDF model and query your graph using Gremlin and SPARQL respectively

Objective Three

Integrate Neptune with services such as Elasticsearch, Glue or Kinesis

• Learn more about Neptune
• Technical resources for developers
• AWS Reference Architectures for Using Graph Databases
• AWS Database blog for Neptune
• Neptune tools and utilities on GitHub

Who Should Watch?

• IT Professionals
• Database Architects

Introduction to Amazon Neptune and its use cases

In this recipe, we will learn about Amazon Neptune. We will also learn about the use cases of Amazon NEptune. Last Updated: 08 Aug 2022

Recipe Objective - Introduction to Amazon Neptune and its use cases?

The Amazon Neptune is a widely used service and is defined as a fully managed graph database service that makes it simple to create and run applications that work with large, interconnected datasets. Amazon Neptune is powered by a purpose-built, high-performance graph database engine that can store billions of relationships and query them in milliseconds. Amazon Neptune supports the popular graph models Property Graph and W3C's RDF, as well as their query languages Apache TinkerPop Gremlin and SPARQL, making it simple to create queries that efficiently navigate highly connected datasets. Recommendation engines, fraud detection, knowledge graphs, drug discovery, and network security are just a few of the graph use cases that Neptune powers. With read replicas, point-in-time recovery, continuous backup to Amazon S3, and replication across Availability Zones, Amazon Neptune is highly available. With support for HTTPS encrypted client connections and encryption at rest, Neptune is safe. Users no longer have to worry about database management tasks like hardware provisioning, software patching, setup, configuration, or backups because Neptune is fully managed. Users don't have to worry about database management tasks like hardware provisioning, software patching, setup, configuration, or backups with Amazon Neptune. Neptune monitors and backs up its database to Amazon S3 in real-time, allowing for granular point-in-time recovery. Amazon CloudWatch can be used to track database performance.

Build a Real-Time Dashboard with Spark, Grafana and Influxdb

Benefits of Amazon Neptune

• Both Gremlin and SPARQL have open graph APIs, and Amazon Neptune provides high performance for both graph models and query languages. It allows users to choose between the Property Graph model and Apache TinkerPop Gremlin, an open source query language, and the W3C standard Resource Description Framework (RDF) model and SPARQL, a standard query language and thus it supports Open graph APIs. Amazon Neptune is a high-performance graph database designed specifically for Amazon. It is designed to handle graph queries. To scale read capacity and execute more than 100,000 graph queries per second, Neptune supports up to 15 low latency read replicas spread across three Availability Zones. As users' needs change, users can easily scale their database deployment from smaller to larger instance types and thus it offers high performance and scalability. Amazon Neptune is highly available, long-lasting, and compliant with the ACID (Atomicity, Consistency, Isolation, and Durability) standards. Neptune is designed to have a 99.99 per cent availability rate. It has fault-tolerant and self-healing cloud storage with six copies of users' data replicated across three Availability Zones. Neptune automatically backs up users' data to Amazon S3 and recovers from physical storage failures in real-time. Instance failover in High Availability typically takes less than 30 seconds and thus it offers high availability and durability. For the user's database, Amazon Neptune provides multiple levels of security, including network isolation via Amazon VPC, support for IAM authentication for endpoint access, HTTPS encrypted client connections, and encryption at rest via Amazon Key Management Service keys users create and control (KMS). Data in the underlying storage, as well as automated backups, snapshots, and replicas in the same cluster, are all encrypted on an encrypted Neptune instance and thus offer security.

System Requirements

• Any Operating System(Mac, Windows, Linux)

This recipe explains Amazon Neptune and the Use cases of Amazon Neptune.

Use cases of Amazon Neptune

• It has a use case in social networking

To create social networking applications, Amazon Neptune can quickly and easily process large sets of user-profiles and interactions. Neptune allows users to integrate social features into your apps by allowing highly interactive graph queries with high throughput. If users are creating a social feed for their app, for example, users can use Neptune to prioritise showing their users the most recent updates from their family, friends whose updates they like,' and friends who live close to them.

• It has a use case for Recommendation engine

Amazon Neptune lets users store relationships between data points like customer interests, friends, and purchase history in a graph and query it quickly to generate personalised and relevant recommendations. For example, users can use Neptune to make product recommendations to a user based on which products have been purchased by others who follow the same sport and have similar purchase histories. Alternatively, users can find people who share a friend but haven't met yet and make a friendship recommendation.

• It has a use case for fraud detection

Users can use Amazon Neptune to process financial and purchase transactions in near real-time, making it easy to spot fraud patterns. Neptune offers a fully managed service that runs fast graph queries to see if a potential buyer is using the same email address and credit card as a previously reported fraud case. If users are working on a retail fraud detection app, Neptune can help them create graph queries to quickly spot patterns like multiple people using the same personal email address or multiple people using the same IP address but living at different physical addresses.

• It has a use case for knowledge graphs

Amazon Neptune aids in the development of knowledge graph applications. A knowledge graph allows users to store data in a graph model and use graph queries to help users navigate large, interconnected datasets. Neptune supports open source and opens standard APIs, allowing users to quickly build knowledge graphs and host them on a fully managed service by leveraging existing information resources. If a user is interested in The Mona Lisa, for example, users can direct them to other works by Leonardo da Vinci or other works of art in The Louvre.

• It has a use case in Life sciences

Amazon Neptune enables users to create life sciences applications that store and navigate data, as well as process sensitive data with ease using encryption at rest. Neptune, for example, can be used to store disease models and gene interactions, as well as to search for graph patterns within protein pathways to discover other genes that may be linked to a disease. Chemical compounds can be represented as graphs, and patterns in molecular structures can be searched for. Neptune can also assist you in integrating data to solve problems in healthcare and life sciences research. Neptune can be used to create and store data across multiple systems, as well as to organise research publications by topic to find relevant information quickly.

What Users are saying..

Abhinav Agarwal

I come from Northwestern University, which is ranked 9th in the US. Although the high-quality academics at school taught me all the basics I needed, obtaining practical experience was a challenge.... Read More

Relevant Projects

Machine learning projects, data science projects, python projects for data science, data science projects in r, machine learning projects for beginners, deep learning projects, neural network projects, tensorflow projects, nlp projects, kaggle projects, iot projects, big data projects, hadoop real-time projects examples, spark projects, data analytics projects for students, you might also like, data science tutorial, data scientist salary, how to become a data scientist, data analyst vs data scientist, data scientist resume, data science projects for beginners, machine learning engineer, pandas dataframe, machine learning algorithms, regression analysis, mnist dataset, data science interview questions, python data science interview questions, spark interview questions, hadoop interview questions, data analyst interview questions, machine learning interview questions, aws vs azure, hadoop architecture, spark architecture.

Deploying auto-reply Twitter handle with Kafka, Spark and LSTM

Yelp Data Processing Using Spark And Hive Part 1

Analyse Yelp Dataset with Spark & Parquet Format on Azure Databricks

Movielens Dataset Analysis on Azure

Build a real-time Streaming Data Pipeline using Flink and Kinesis

Snowflake Real Time Data Warehouse Project for Beginners-1

AWS Project for Batch Processing with PySpark on AWS EMR

AWS Project - Build an ETL Data Pipeline on AWS EMR Cluster

Build a Data Pipeline with Azure Synapse and Spark Pool

SQL Project for Data Analysis using Oracle Database-Part 2

Subscribe to recipes, sign up to view recipe.

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications You must be signed in to change notification settings

aws-samples/amazon-neptune-ml-fake-news-detection

Folders and files, repository files navigation, detecting social media fake news with amazon neptune ml.

The spread of misinformation and fake news on social media has posed a major challenge to the well-being of individuals and societies. It is imperative that we develop robust and automated solutions for early detection of fake news on social media. Traditional approaches rely purely on the news content to mark an information piece as real or fake news. However, the social context in which the news is published and spread can provide additional insights into the nature of fake news on social media and hence improve the predictive capabilities of fake news detection tools. Here we demonstrate how Amazon Neptune ML can be leveraged to detect fake news based on the content and social context of the news spread on social media.

We use the BuzzFeed dataset from the 2018 version of FakeNewsNet in this work. The BuzzFeed dataset is made of a sample of news articles shared on Facebook from 9 news agencies over one week leading up to the 2016 U.S. election. Every post and the corresponding news article have been fact-checked by BuzzFeed journalists. Table below summarizes key statistics about the dataset:

We process the raw data from FakeNewsNet repository; and convert them into comma-separated-values (CSV) format for vertices (nodes) and edges in a heterogeneous property graph that can be readily loaded into an Amazon Neptune database with Apache TinkerPop Gremlin. The constructed property graph is composed of 4 vertex types and 5 edge types, as demonstrated in the schematic below, which together describe the social context in which each news is published and spread.

Graph Machine Learning with Neptune ML

Graph Machine Learning with Amazon Neptune ML involves five main steps:

Data Export and Configuration – The data-export step uses the Neptune-Export service to export data from Neptune into Amazon S3 in CSV format. A configuration file named training-data-configuration.json is automatically generated which specifies how the exported data can be loaded into a trainable graph.

Data Preprocessing – The exported dataset is preprocessed using standard techniques to prepare it for model training. Feature normalization can be performed for numeric data, and text features can be encoded using word2vec. At the end of this step, a DGL (Deep Graph library) graph is generated from the exported dataset for the model training step. This step is implemented using a SageMaker processing job, and the resulting data is stored in an Amazon S3 location that you have specified.

Model Training – This step trains the machine learning model that will be used for predictions. Model training is done in two stages: The 1st stage uses a SageMaker processing job to generate a model training strategy configuration set that specifies what type of model and model hyperparameter ranges will be used for the model training. The 2nd stage uses a SageMaker model tuning job to try different hyperparameter configurations and select the training job that produced the best-performing model. The tuning job runs a pre-specified number of model training job trials on the processed data. At the end of this stage, the trained model parameters of the best training job are used to generate model artifacts for inference.

Creating Inference Endpoint in Amazon SageMaker – The inference endpoint is a SageMaker endpoint instance that is launched with the model artifacts produced by the best training job. The endpoint is able to accept incoming requests from the graph database and return the model predictions for inputs in the requests.

Querying the ML Model using Gremlin – You can use extensions to the Gremlin query language to query predictions from the inference endpoint.

Running Code

Deploy the infrastructure.

Note: Use this CDK code to quickly spin up the required resources, including:

• an Amazon Sagemaker notebook where you can run this code
• an associated Amazon Neptune cluster
• all the configurations needed to work with Neptune ML in the Amazon Sagemaker.

The CDK code is specifically designed to work for this repository, so you shouldn't have any errors when running the notebooks. Note that to run the 1-create-graph-dataset.ipynb notebook, a NotebookInstanceType of ml.c5.9xlarge is configured already to ensure you have enough memory to load the UserFeature.mat file.

Clone this into the notebook instance

Once you have deployed the above CDK code,

• navigate to the Sagemaker console
• expand the Notebook section on the left hand side.
• Click notebook instances
• Find the notebook deployed by the CDK. It should start with aws-neptune-notebook-for-neptunedbcluster . If you don't see any, check to ensure you are in the correct region and the CDK has finished deploying.
• Click Open Jupyer Lab
• Launch a new terminal
• Run cd Sagemaker && git clone https://github.com/aws-samples/amazon-neptune-ml-fake-news-detection.git
• You should see the folder amazon-neptune-ml-fake-news-detection on the left hand side
• Click into the folder, and you will see the notebooks described below as well as additional files and folders. Follow the steps below to run the notebooks.

Run the notebooks

• As an initial step, run through 1-create-graph-dataset.ipynb notebook to generate the graph dataset.
• Use the 2-load-graph-dataset.ipynb notebook to load the graph dataset to an Amazon Neptune cluster.
• Use the 3-detect-fake-news-neptune-ml.ipynb notebook to run through steps 1 to 5 above of graph machine learning with Amazon Neptune ML an interactive way.
• Use the 4-inductive-inference.ipynb notebook to go through an example of real-time inductive inference with Amazon Neptune ML.

If needed, you can run the for-reference_clear-neptune-database.ipynb notebook to clear the Amazon Neptune database. This can be helpful if you run the sample and then want to create your own data in Amazon Neptune.

See CONTRIBUTING for more information.

This library is licensed under the MIT-0 License. See the LICENSE file.

Code of conduct

Security policy, contributors 4.

• Jupyter Notebook 64.5%
• Python 35.5%

What Is Amazon Neptune?

Amazon Neptune is a fast, reliable, fully managed graph database service that makes it easy to build and run applications that work with highly connected datasets. The core of Neptune is a purpose-built, high-performance graph database engine. This engine is optimized for storing billions of relationships and querying the graph with milliseconds latency. Neptune supports the popular property-graph query languages Apache TinkerPop Gremlin and Neo4j's openCypher, and the W3C's RDF query language, SPARQL. This enables you to build queries that efficiently navigate highly connected datasets. Neptune powers graph use cases such as recommendation engines, fraud detection, knowledge graphs, drug discovery, and network security.

The Neptune database is highly available, with read replicas, point-in-time recovery, continuous backup to Amazon S3, and replication across Availability Zones. Neptune provides data security features, with support for encryption at rest and in transit. Neptune is fully managed, so you no longer need to worry about database management tasks like hardware provisioning, software patching, setup, configuration, or backups.

Neptune Analytics; is an analytics database engine that complements Neptune database and that can quickly analyze large amounts of graph data in memory to get insights and find trends. Neptune Analaytics is a solution for quickly analyzing existing graph databases or graph datasets stored in a data lake. It uses popular graph analytic algorithms and low-latency analytic queries.

To learn more about using Amazon Neptune, we recommend that you start with the following sections:

Getting started with Amazon Neptune

Overview of Amazon Neptune features

Also, before you begin designing a database, we recommend that you consult the GitHub repository Amazon Reference Architectures for Using Graph Databases , where you can inform your choices about graph data models and query languages, and browse examples of reference deployment architectures.

Key Service Components

Primary DB instance – Supports read and write operations, and performs all of the data modifications to the cluster volume. Each Neptune DB cluster has one primary DB instance that is responsible for writing (that is, loading or modifying) graph database contents.

Neptune replica – Connects to the same storage volume as the primary DB instance and supports only read operations. Each Neptune DB cluster can have up to 15 Neptune Replicas in addition to the primary DB instance. This provides high availability by locating Neptune Replicas in separate Availability Zones and distribution load from reading clients.

Cluster volume – Neptune data is stored in the cluster volume, which is designed for reliability and high availability. A cluster volume consists of copies of the data across multiple Availability Zones in a single Amazon Region. Because your data is automatically replicated across Availability Zones, it is highly durable, and there is little possibility of data loss.

Supports Open Graph APIs

Amazon Neptune supports open graph APIs for both property graphs (Gremlin and openCypher) and RDF graphs (SPARQL). It provides high performance for both of these graph models and their query languages. You can choose the Property Graph (PG) model and access the same graph with both the openCypher query language and/or the Gremlin query language . If you use the W3C standard Resource Description Framework (RDF) model, you can access your graph using the standard SPARQL query language .

Highly Secure

Neptune provides multiple levels of security for your database. Security features include network isolation using Amazon VPC , and encryption at rest using keys that you create and control through Amazon Key Management Service (Amazon KMS) . On an encrypted Neptune instance, data in the underlying storage is encrypted, as are the automated backups, snapshots, and replicas in the same cluster.

Fully Managed

With Amazon Neptune, you don’t have to worry about database management tasks like hardware provisioning, software patching, setup, configuration, or backups.

You can use Neptune to create sophisticated, interactive graph applications that can query billions of relationships in milliseconds. SQL queries for highly connected data are complex and hard to tune for performance. With Neptune, you can use the popular graph query languages Gremlin, openCypher, and SPARQL to execute powerful queries that are easy to write and perform well on connected data. This capability significantly reduces code complexity so that you can quickly create applications that process relationships.

Neptune is designed to offer greater than 99.99 percent availability. It increases database performance and availability by tightly integrating the database engine with an SSD-backed virtualized storage layer that is built for database workloads. Neptune storage is fault-tolerant and self-healing. Disk failures are repaired in the background without loss of database availability. Neptune automatically detects database crashes and restarts without the need for crash recovery or rebuilding the database cache. If the entire instance fails, Neptune automatically fails over to one of up to 15 read replicas.

Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Amazon Neptune ML for machine learning on graphs

There is often valuable information in large connected datasets that can be hard to extract using queries based on human intuition alone. Machine learning (ML) techniques can help find hidden correlations in graphs with billions of relationships. These correlations can be helpful for recommending products, predicting credit worthiness, identifying fraud, and many other things.

The Neptune ML feature makes it possible to build and train useful machine learning models on large graphs in hours instead of weeks. To accomplish this, Neptune ML uses graph neural network (GNN) technology powered by Amazon SageMaker and the Deep Graph Library (DGL) (which is open-source ). Graph neural networks are an emerging field in artificial intelligence (see, for example, A Comprehensive Survey on Graph Neural Networks ). For a hands-on tutorial about using GNNs with DGL, see Learning graph neural networks with Deep Graph Library .

Graph vertices are identified in Neptune ML models as "nodes". For example, vertex classification uses a node-classification machine learning model, and vertex regression uses a node-regression model.

What Neptune ML can do

Neptune supports both transductive inference, which returns predictions that were pre-computed at the time of training, based on your graph data at that time, and inductive inference, which returns applies data processing and model evaluation in real time, based on current data. See The difference between inductive and transductive inference .

Neptune ML can train machine learning models to support five different categories of inference:

Types of inference task currently supported by Neptune ML

Node classification   –   predicting the categorical feature of a vertex property.

For example, given the movie The Shawshank Redemption , Neptune ML can predict its genre property as story from a candidate set of [story, crime, action, fantasy, drama, family, ...] .

There are two types of node-classification tasks:

Single-class classification : In this kind of task, each node has only one target feature. For example, the property, Place_of_birth of Alan Turing has the value UK .

Multi-class classification : In this kind of task, each node can have more than one target feature. For example, the property genre of the film The Godfather has the values crime and story .

Node regression   –   predicting a numerical property of a vertex.

For example, given the movie Avengers: Endgame , Neptune ML can predict that its property popularity has a value of 5.0 .

Edge classification   –   predicting the categorical feature of an edge property.

There are two types of edge-classification tasks:

Single-class classification : In this kind of task, each edge has only one target feature. For example, a ratings edge between a user and a movie might have the property, liked , with a value of either "Yes" or "No".

Multi-class classification : In this kind of task, each edge can have more than one target feature. For example, a ratings between a user and movie might have multiple values for the property tag such as "Funny", "Heartwarming", "Chilling", and so on.

Edge regression   –   predicting a numerical property of an edge.

For example, a rating edge between a user and a movie might have the numerical property, score , for which Neptune ML could predict a value given a user and a movie.

Link prediction   –   predicting the most likely destination nodes for a particular source node and outgoing edge, or the most likely source nodes for a given destination node and incoming edge.

For example, with a drug-disease knowledge graph, given Aspirin as the source node, and treats as the outgoing edge, Neptune ML can predict the most relevant destination nodes as heart disease , fever , and so on.

Or, with the Wikimedia knowledge graph, given President-of as the edge or relation and United-States as the destination node, Neptune ML can predict the most relevant heads as George Washington , Abraham Lincoln , Franklin D. Roosevelt , and so on.

Node classification and Edge classification only support string values. That means that numerical property values such as 0 or 1 are not supported, although the string equivalents "0" and "1" are. Similarly, the Boolean property values true and false don't work, but "true" and "false" do.

With Neptune ML, you can use machine learning models that fall in two general categories:

Types of machine learning model currently supported by Neptune ML

Graph Neural Network (GNN) models   –   These include Relational Graph Convolutional Networks (R-GCNs) . GNN models work for all three types of task above.

Knowledge-Graph Embedding (KGE) models   –   These include TransE , DistMult , and RotatE models. They only work for link prediction.

User defined models   –   Neptune ML also lets you provide your own custom model implementation for all the types of tasks listed above. You can use the Neptune ML toolkit to develop and test your python-based custom model implementation before using the Neptune ML training API with your model. See Custom models in Neptune ML for details about how to structure and organize your implementation so that it's compatible with Neptune ML's training infrastructure.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Thanks for letting us know we're doing a good job!

If you've got a moment, please tell us what we did right so we can do more of it.

Thanks for letting us know this page needs work. We're sorry we let you down.

If you've got a moment, please tell us how we can make the documentation better.

• About Amazon (English)
• About Amazon (日本語)
• About Amazon (Français)
• About Amazon (Deutsch)
• Newsroom (Deutsch)
• About Amazon (Italiano)
• About Amazon (Polski)
• About Amazon (Español)
• Press Center (English)
• Press Center
• About Amazon (Português)
• Press Release Archive
• Images & Videos
• Investor Relations

AWS Announces General Availability of Amazon Neptune

Amazon Neptune, a fast and reliable graph database, makes it easy for customers to build applications on highly connected datasets

Thousands of customers, including Samsung Electronics, Pearson, Intuit, Siemens, AstraZeneca, FINRA, LifeOmic, Blackfynn, and Amazon Alexa, participated in the preview, building new graph applications and battle-testing their production workloads

SEATTLE--(BUSINESS WIRE)--May 30, 2018-- Today, Amazon Web Services, Inc. (AWS), an Amazon.com company (NASDAQ: AMZN), announced general availability of Amazon Neptune, a fast, reliable, and fully managed graph database service. Amazon Neptune efficiently stores and navigates highly connected data, allowing developers to create sophisticated, interactive graph applications that can query billions of relationships with millisecond latency. In the preview, customers used Neptune to build social networks, recommendation engines, fraud detection, knowledge graphs, drug discovery applications, and more. With Amazon Neptune there are no upfront costs, licenses, or commitments required; customers pay only for the Neptune resources they use. To get started with Amazon Neptune, visit  https://aws.amazon.com/neptune .

For many years, the relational database was the only option available to application developers. While relational databases are great for applications that need referential integrity for highly structured data, today’s developers need a variety of databases to serve the needs of modern applications. Just as some modern applications need to store petabytes of unstructured data, access it with sub-millisecond latency, process millions of requests per second, and scale to support millions of users all around the world, others need to quickly understand and navigate billions of relationships between highly connected data. Applications (e.g. social applications, recommendation engines, and fraud detection) that try to navigate these relationships using SQL queries require join operations that quickly become complex, error prone, and slow to execute. Graph databases are becoming more attractive because they store the relationships between highly connected data as graphs, enabling applications to quickly query these relationships. However, many of today’s open source or community edition graph databases can’t offer the performance, reliability or scalability of commercial databases, while commercial options are often expensive, or complex to set up and maintain. Now, with Amazon Neptune, developers can query connected datasets with the speed and simplicity of a graph database, while benefiting from the scalability, security, durability, and availability of an AWS managed graph database service. The Amazon Neptune query processing engine is optimized for both of the leading graph models, Property Graph and W3C's RDF, and their associated query languages, Apache TinkerPop Gremlin and RDF/SPARQL, providing customers the flexibility to choose the right approach based on their specific graph use case. And, as a customer’s data increases, Neptune storage scales automatically, without downtime or performance degradation.

“The days of modern technology companies using relational databases for all of their workloads have come and gone,” said Raju Gulabani, Vice President, Databases, Analytics, and Machine Learning at Amazon Web Services, Inc. “As the world has become more connected, applications that navigate large, connected datasets are increasingly more critical for customers. We are delighted to give customers a high-performance graph database service that enables developers to query billions of relationships in milliseconds using standard APIs, making it easy to build and run applications that work with highly connected datasets.”

Amazon Neptune is highly available and durable, automatically replicating six copies of data across three Availability Zones and continuously backing up data to Amazon Simple Storage Service (Amazon S3). Amazon Neptune is designed to offer greater than 99.99 percent availability and automatically detects and recovers from most database failures in less than 30 seconds. Amazon Neptune also provides advanced security capabilities, including network security through Amazon Virtual Private Cloud (VPC), and encryption at rest using AWS Key Management Service (KMS).

Pearson, the world’s learning company, helps more than 75 million people each year embrace their passion and ambition for learning new skills. “Our goal is to significantly reduce complexity in several of our key systems and to standardize how our organization approaches heavily connected, graph data,” said Josh Walton, Senior Solutions Architect of Pearson. “We’ve long wanted to use a graph database, but until we evaluated Neptune, nothing fit our needs due to performance or cost. We tried Neptune during the preview and the promise it shows is exciting. Our performance test results have been exceptional and the pricing is right. We expect to be able to simplify our existing architecture and maintain current SLAs while also gaining the ability to scale beyond our current capacity, and speed up development time with all the packaged, fully managed goodies that come along with AWS managed services.”

Blackfynn is a life sciences software startup formed to change how Epilepsy, Alzheimer’s disease, Parkinson’s disease, ALS, and other neurological diseases are treated. The multidisciplinary team has deep roots in medicine, clinical development, neuroscience, engineering, data science and machine learning. “We look forward to using Amazon Neptune as an integral part of our data platform,” said Chris Baglieri, SVP of Engineering at Blackfynn. “Neptune will allow us to connect the dots between genomics, pathology, neurochemistry, device and patient clinical data, efficiently and at scale, helping us drive breakthrough discoveries.”

Intuit is powering prosperity for millions of small businesses and self-employed customers around the world. “We are exploring Amazon Neptune for heavy lifting of our Graph infrastructure management, freeing our engineering team to focus on optimizing the knowledge graph for customer-centric business problems,” said Sandeep Uttamchandani, Chief Data Architect, Intuit Small Business Group and Self-Employed Group. “Neptune is promising as a secure and highly available Graph database available out of the box!”

FINRA regulates a critical part of the securities industry – brokerage firms doing business with the public in the United States. FINRA takes in up to 100 billion market events per day that are tracked, aggregated, and analyzed for the purpose of protecting investors. “FINRA is in the process of migrating most of our databases to AWS,” said Saman Michael Far, Senior Vice President, FINRA. “We have evaluated Amazon Neptune, and we look forward to beginning to use it for our graph workloads.”

LifeOmic leverages the cloud, machine learning and mobile devices to enable precision medicine for healthcare providers, researchers, health IT companies and patients. LifeOmic’s JupiterOne is a DevSecOps solution built in the cloud, for the cloud to automate and simplify security and compliance such as HIPAA and HITRUST. “Amazon Neptune has accelerated the development of our JupiterOne DevSecOps software while reducing our operational costs, enabling us to bring enterprise grade security solutions to companies of all sizes,” said Erkang Zheng, Chief Information Security Officer at LifeOmic. “Security operations, analytics and compliance fundamentally boil down to the ability to ask the right questions and get the right answers quickly, but the speed of DevOps and the ephemeral nature of cloud make this difficult. With Neptune, we can harness the power of graph databases without infrastructure maintenance overhead. This enables us to focus on helping organizations build solutions that simplify security and fast track compliance and certifications such as HIPAA, HITRUST and NIST.”

“Amazon Neptune is a key part of the toolkit we use to continually expand Alexa’s knowledge graph for our tens of millions of Alexa customers—it’s just Day 1 and we’re excited to continue our work with the AWS team to deliver even better experiences for our customers,” said David Hardcastle, Director of Amazon Alexa, Amazon.

Amazon Neptune is available in the US East (N. Virginia), US East (Ohio), US West (Oregon), and EU (Ireland) regions, and will expand to additional Regions in the coming year.

About Amazon Web Services

For over 12 years, Amazon Web Services has been the world’s most comprehensive and broadly adopted cloud platform. AWS offers over 125 fully featured services for compute, storage, databases, networking, analytics, machine learning and artificial intelligence (AI), Internet of Things (IoT), mobile, security, hybrid, virtual and augmented reality (VR and AR), media, and application development, deployment, and management from 55 Availability Zones (AZs) within 18 geographic regions and one Local Region around the world, spanning the U.S., Australia, Brazil, Canada, China, France, Germany, India, Ireland, Japan, Korea, Singapore, and the UK. AWS services are trusted by millions of active customers around the world—including the fastest-growing startups, largest enterprises, and leading government agencies—to power their infrastructure, make them more agile, and lower costs. To learn more about AWS, visit  https://aws.amazon.com .

About Amazon

Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. Customer reviews, 1-Click shopping, personalized recommendations, Prime, Fulfillment by Amazon, AWS, Kindle Direct Publishing, Kindle, Fire tablets, Fire TV, Amazon Echo, and Alexa are some of the products and services pioneered by Amazon. For more information, visit  www.amazon.com/about  and follow  @AmazonNews .

View source version on businesswire.com:  https://www.businesswire.com/news/home/20180530006462/en/

Source: Amazon.com, Inc.

Amazon.com, Inc. Media Hotline [email protected] www.amazon.com/pr

Advertisement

How Hunter tracked down piracy with Tootega Nahi Todega

• Facebook Messenger

By The Drum, Editorial

May 30, 2024 | 4 min read

Listen to article 4 min

Winning Gold in The Drum Awards for Marketing APAC’s Social Media category is ‘Hunter hunts down the piracy scenes with Tootega Nahi Todega’ from Amazon miniTV India & SoCheers. Here is the award-winning case study.

Our challenge: launching the show on a platform that usually gets romance-focused viewers and reviving interest in a popular 90s Bollywood actor, Suniel Shetty.

Our response to the challenge: let’s do something pathbreaking.

How did we go about it?

Breaking genre norms, breaking the web.

Amazon miniTV is a platform that provides free content across genres, inbuilt into the Amazon.in app in India. People in India love free content and hence flock to sites, platforms, or links that provide it.

Using that insight, we bypassed traditional social media channels, directly targeting Torrent and Telegram, platforms synonymous with pirated content. Users searching for the latest popular movies encountered a surprise – an interactive video featuring Suniel Shetty in his character from the latest Amazon miniTV show, Hunter . With a clever twist on his iconic dialogue, Shetty informed viewers he had “broken the link to free content” and urged them to instead discover high-quality, free content on Amazon miniTV.

Beyond the Click: A Multi-Sensory Experience

The campaign’s brilliance extended beyond the video. Hoardings featuring Shetty’s character, portraying him as the ‘breaker of illegal things,’ were strategically placed throughout cities, further amplifying the intrigue. Meticulously seeded content online added another layer to this multi-sensory experience.

Results that Shattered Expectations

This unconventional approach yielded phenomenal results:

• 671,000+ Engagements: The campaign sparked significant conversation and excitement over a short span of two weeks.
• 9.4m+ Views: Our message reached a vast audience.
• 20,000 Clicks on Torrent & Telegram: Users actively engaged, demonstrating the campaign’s effectiveness in driving viewers to the platform.

Why do we deserve to win?

The Hunter campaign was more than numbers: a blueprint for OTT marketing.

Hunter’s social media success wasn’t just about statistics. It redefined how action-thrillers are promoted on OTT platforms. Here’s what made this campaign stand out:

• Unique Platform Targeting: By directly addressing audience habits and disrupting their content consumption patterns, SoCheers reached viewers where they were.
• Storytelling & Nostalgia: We leveraged the narrative and the return of a 90s action hero to create a story that resonated with audiences.
• Interactive Elements: The “Tootega Nahi Todega” campaign and broken link seeding actively engaged viewers, blurring the lines between reality and fiction.
• Strategic Collaborations: Partnerships with relevant brands and influencers broadened our reach and amplified the campaign message.
• Genre-Specific Resonance: Leveraging Shetty’s action-hero persona perfectly aligned with the show’s genre and resonated with the target audience.

Hunter’s social media campaign is a masterclass in OTT marketing innovation. We combined captivating storytelling, nostalgia, interactive elements, and strategic partnerships to achieve remarkable results. This campaign serves as a blueprint for future shows, demonstrating the transformative power of social media in the OTT entertainment industry.

Ready to get your work recognized on a global stage? Enter The Drum Awards today . Need more inspiration, read our Award Winning Case Studies.

More from Awards Case Studies