WEB SERVICE APIs
The BeliefNetworks REST API provides semantic concepts, web site recommendations, Twitter site recommendations, and custom recommendations based on URL or raw text input.
Contents
Overview — It's about The Concepts
The BeliefNetworks service discovers concepts locked within diverse data types such as unformatted text documents, HTML documents, Word documents, PDF files, RSS feeds and XML representations dumped from databases. Automating this simple idea through our DataChasm platform allows us to provide you with the foundation for your next-generation, intelligent web application.
Concept discovery is fundamental to the BeliefNetworks service. If you are familiar with search text discovery; consider a starting point of "I'm a little teapot". As you can see, keywords, in this context, have no relevance.
However, that's just scratching the surface. Once DataChasm has mined and conceptually mapped a document's concepts, the document becomes a resource for richer search features. Up until now, conceptually mapped documents were typically endpoints of a search scenario; in other words, a document to be discovered. However, BeliefNetworks applies an interesting twist by using the conceptually mapped document as the search starting point. For example, a conceptually mapped document can be the target of the statement "Find me all documents that are like this document". As soon as you consider that documents don't have to be literally documents, that they can also represent anything in your domain, the possibilities begin to multiply.
Finding Not Searching
Imagine you’re trying to find all documents related to a document about "The History of Military Aircraft" using a straight-up search appliance. First, you would likely start a search with the phrase “The History of Military Aircraft”. We've already seen an example of the subtle differences between discovery of search keywords and the discovery of higher-level conceptual meaning. What can you expect? You'll probably find a lot of documents about the exact topics of military aircraft and it’s history. However you’re unlikely to find documents on related concepts such as the history of flight, the use of gliders to teach Air Force Academy cadets to fly, or the little known female test pilots of WWII.
Using a BeliefNetworks' powered application, discovery of documents related to the History of Military Aircraft can be as simple as "pushing a button". The DataChasm platform automates the separate processes of concept extraction and related document search that you would otherwise have to do manually.
Recommendations based on what you know
That was a discovery scenario. Let's think about a recommendation scenario. Suppose you are building an application, the premise of which is that you can provide value to a user base by matching pet owners to pet service providers (groomers, walkers, stables, supplies vendors, etc). Both pet owners and pet service providers can register in your system, but they are two separate groups that are managed separately by your application.
There are affinities between the two groups that, if discovered, may lead to greater user satisfaction with your service, good buzz, more customers, and ultimately higher profits for you. For instance, obviously you don't recommend a stable to someone looking for a kennel, but a person looking for a kennel for Roscoe for the weekend may have some particular likes and needs that can be matched to the right kennel with minimal effort for Roscoe's owner.
In this case, the "documents" scanned for concepts are pet owner profiles they freely enter at registration time describing themselves and their pets (with appropriate confidence the details won't be sold or revealed). Profiles would be fed, as created, into the conceptual mapper. Meanwhile, pet service providers would fill out some forms describing their offerings that ultimately could be rendered as XML documents. As you can see, there are a number of approaches for each major actor in your application.
Now, imagine a pet-services discovery service powered by BeliefNetworks. The conceptual information for all pet owners is mapped and placed into one silo, and likewise another silo of conceptual information is built for all pet service providers. Finally, we return to Roscoe's owner who pushes a button on the BeliefNetworks' powered application to start searching for recommendations in the pet services silo based upon the conceptual information gleaned from their owner's profile.
It could also work another way. Suppose you offer anonymous direct marketing, something that would inform pet owners of goods and services but would not expose their personal data to the service providers. This approach would not be as annoying as a phone call, but could be a message on their customer page linking them to the service provider that they can click on if they're interested. Conversely, the pet service provider, interested in buying a batch of push marketing, pushes the BeliefNetworks powered button in your service to discover pet owners most likely to be interested in their products and services by matching the pet service provider's profile to the most conceptually relevant profiles of pet owners.
The DataChasm Service
Version 1.0 of BeliefNetworks’ DataChasm web service provides RESTful programmatic conceptual discovery against two large data repositories — The World Wide Web and Twitter. No additional effort is required on the part of the programmer other than to make the call. These two data repositories are most useful for interesting search applications and mashups. As an example, the MashMeUp web application illustrates the functionality of concept discovery with Twitter and web site recommendations.
The DataChasm web service also allows you to build your own silos of conceptual information, closely tuned to your own application business needs. Once built, those silos become the foundation for recommendation features, that until now, you've only wished you could have.
But, to use the APIs properly, some conceptual information should be introduced (pun intended).
Concepts, Concepts, Concepts
Concepts are key to successfully using the DataChasm service. We like to think of concepts as information thumbnails (actually, we consider them a higher cognitive learning process, but that is another discussion for another day — we usually provide the beer for that one). Simple keyword search, while actionable by the APIs, isn’t as successful as starting with a document that expresses and represents the concepts you are interested in discovering. You'll find many APIs accept a generic "text" parameter. This may refer to as little or as much text as you need to represent your concepts, or it can refer to a fully qualified URL which points to a document or resource on the web (think article, publication, XML output from some other API) that encapsulates the concepts. For instance, you may have better results pointing to an article on The Cramps for a punk band search than simply the text "The Cramps".
Recommendations
A recommendation is the relationship between your starting point (text or URL) and the haystack of information with which you wish to establish an affinity in order to discover the "needles", sorted for relevance. What BeliefNetworks believes to be the most relevant possibilities are returned first; if you wish, you can go deeper into the "stack" with successive queries. For instance, when you ask for recommendations from Twitter, you are returned ongoing conversations considered most relevant to your conceptual starting point. We like to call that "tweaning"; in other words, bringing meaning to tweeting.
"Weight" is the term we use for the degree of relevance a recommendation has to your chosen starting point. You'll see various weights returned for recommendations, and you can build logic around this such as sorting, cut-off, etc.
Custom APIs
Our custom APIs allow you to build applications against your own business data. You can define silos of information by business need, seed them and maintain them with resources, and then query them for recommendations, just like you can with our Twitter and Web APIs. This is a powerful tool that allows you to discover conceptual relationships of the highest relevance against between your own repositories of business data.
One Corpus, Many Corpora
The "Corpora" referred to by BeliefNetworks' APIs are your information silos. You create a corpus to hold the results of resources that are related by conceptual consistency. For example, if you were Amazon, one corpus might hold Books, while another might hold Reviewers. A resource list is typically provided at creation time. The corpus resources are subsequently managed and updated as resources come and go. Every 24 hours, DataChasm crawls to discover any new resources that should be added and removes all those marked for deletion your corpus.
Once a corpus is created, you may begin to query it for recommendations.
From Here …
From here you should review the Core APIs and the Custom APIs for detailed information on how to use the service as well as for specific examples. The examples tie it all together with several examples in various languages — Java, Ruby, C# — to name a few. Review our Security policy to understand the requirements for securing your data, then go to the Get Started page to register an API key.
Then … experiment! See what discoveries are awaiting your application!