got data?

got data?

For some amount of time (ie., until the bandwidth costs add up 🙂 ) you can download the figures extracted from arxiv documents, during the development and testing of the search API described in prior posts. If you have the AWS CLI installed, getting the figure meta data (from which you can create download URLs)Continue Reading got data?

new api method for data image search

new api method for data image search

As mentioned, after doing some experiments for the KDnuggets article, I bundled some of the existing API methods into a new one, which will extract from a page figures that have x/y scaling information. The JSON output is well suited to elasticsearch or your favorite flavor of NoSQL eg., an extract of the response: [{“input”:”tmp/quant-ph0002044-9.png”,Continue Reading new api method for data image search

kdnuggets

kdnuggets

We’re very grateful to Dr Piatetsky-Shapiro for the chance to publish an item in kdnuggets, check it out here. In the process of putting some examples together for the article, I think I’ve finally landed on a useful workflow and schema for the figure data search engine, hoping to get that out asap; stay tunedContinue Reading kdnuggets

arxiv mining

arxiv mining

Some time ago I launched a little project, mining data from arxiv; you can read about it in other blog posts. Specifically, I modeled figures from about 500k figures as Gaussian mixture models, in order to create some features, so figures might be ultimately represented as graphs for comparison. More ordinary methods might suffice tooContinue Reading arxiv mining

DevOps with AWS

DevOps with AWS

One of the distinct advantages of devOps philosophy as implemented using AWS is the ability to rapidly kick out a new service. The new template-match API is a good example. Knowing the algorithm, I was able to quickly pull something together from the p2t C++ library, an example input image attached. The algorithm determines whereContinue Reading DevOps with AWS