Python API clients with Tapioca: simplify integrations

In this post I'll present to you <a href="http://tapioca-wrapper.readthedocs.io/">Tapioca</a>, a Python library to create powerful API clients with very few lines of code. If you don't want to read through the reasons why I've built it, you may just jump straight to the <a href="https://www.vinta.com.br/blog/2016/python-api-clients-with-tapioca/#tapioca-wrapper">Tapioca Wrapper</a> section.<h2 id="why-do-we-need-a-better-way-to-build-api-clients">Why do we need a better way to build API clients</h2>Integrating with external services is painful. Here at Vinta, we build and deal with both public and private APIs on the daily basis. Having worked in many of those projects I can safely tell that we are currently in hell when it comes to integrations. There are two ends in this mess: one is the result of poorly designed APIs and the other is the lack of standards and good practices for building API clients.<figure class="kg-card kg-image-card kg-card-hascaption"><img src="https://vinta-cms.s3.amazonaws.com/media/filer_public/4a/a3/4aa3e3eb-e52b-460f-a1ab-6d11bcbb455c/api.png" class="kg-image"><figcaption><a href="https://xkcd.com/1481/">https://xkcd.com/1481/</a></figcaption></figure>It begins with API design. As we get away from RPC based architectures and move to more REST style systems, there is clearly a lack of well formed practical set of rules or at least a broad misunderstanding of the rules for building APIs. This is actually not a surprise since we were used to RPC and its well defined protocols while REST provides a set of guides, and is actually an architectural style. Luckily there are people trying to solve this problem, the <a href="http://jsonapi.org/">json:api</a> project is a good example of where we should aim for good design and standardization.<figure class="kg-card kg-image-card kg-card-hascaption"><img src="https://vinta-cms.s3.amazonaws.com/media/filer_public/8b/ab/8bab2ae1-31e5-4e4a-8d7d-64c466f82af6/coxfmazxaaatv9a.jpg" class="kg-image"><figcaption>Roy Fielding, creator of the REST architectural style</figcaption></figure>On the other side there are people trying to use these poorly designed APIs. Most developers doing integrations don't want to deal with the complicated and unique patterns API creators freely chose for their systems. The easiest [and most common] way of trying to stay away from this mess is to use open source available API clients (or API wrappers). Which sounds like a good plan as long as you are doing a simple integration with a single service. As you move to multiple complex integrations you start noticing that the API client spectrum is as messy as the API services. Every single wrapper lib has it own unique way of dealing with authentication, pagination, versioning, throttling and parameter passing. Not to mention the huge amount of duplicated code to perform the exact same set of features among them.While we developers using public APIs do not have control over how they are built, we do have on the API client side. This takes us to Tapioca.<h2 id="tapioca-wrapper">Tapioca Wrapper</h2><figure class="kg-card kg-image-card"><img src="https://vinta-cms.s3.amazonaws.com/media/filer_public/70/ce/70cea57c-5f0f-48b0-a77d-44c0316f73f0/tapioca-logo.png" class="kg-image"></figure><a href="http://tapioca-wrapper.readthedocs.io/en/stable/quickstart.html">Tapioca</a> is a Python API client maker. It gathers most of the features API clients implement and puts them in an extensible core. Wrappers will then extend this core implementing only the specifics from each service (such as authentication and pagination) and get all the common API client features for free. Tapioca approach also comes in handy because regardless of the service, clients look the same in the way you interact with them. Once you've worked with a Tapioca lib you know how to use them all, no need to learn things from scratch.Here is the <a href="https://github.com/vintasoftware/tapioca-facebook"><code>tapioca-facebook</code></a> in action.<pre><code>>>> from tapioca_facebook import Facebook >>> >>> api = Facebook(access_token='{your-access-token}') >>> feed = api.user_feed(id='me').get() >>> print(feed.data[0].link) <TapiocaClient object 'https://www.youtube.com/watch?v=9D_q-8m_4zw'> </code></pre>And now the <a href="https://github.com/vintasoftware/tapioca-twitter"><code>tapioca-twitter</code></a>:<pre><code>>>> from tapioca_twitter import Twitter >>> >>> api = Twitter( >>> api_key='{your-api-id}', >>> api_secret='{your-api-secret}', >>> access_token='{your-access-token}', >>> access_token_secret='{your-access-token-secret}') >>> >>> statuses = api.statuses_user_timeline().get() >>> print(statuses[0].text) <TapiocaClient object ('RT @vintasoftware: Our latest blog post was featured in @thedjangoweekly and ' '@importpython newsletters! Go check it out: https://t.co/CCnOVtGr8V')> </code></pre>Pagination is also very easy and similar for both services.Facebook:<pre><code>>>> feed = api.user_feed(id='me').get() >>> for item in feed().pages(): >>> print(item.id) ... </code></pre>Twitter:<pre><code>>>> statuses = api.statuses_user_timeline().get() >>> for item in statuses().pages(): >>> print(item.id) ... </code></pre>There are many other features (e.g.: serialization) you can check <a href="http://tapioca-wrapper.readthedocs.io/en/stable/features.html">here</a>.To give you an idea on how easy it is to build a client with Tapioca, here is the source code for the <code>tapioca-facebook</code> we just played with. <a href="https://github.com/vintasoftware/tapioca-facebook/blob/master/tapioca_facebook/tapioca_facebook.py">See the code in the repository</a>.<pre><code>from tapioca import ( TapiocaAdapter, generate_wrapper_from_adapter, JSONAdapterMixin) from requests_oauthlib import OAuth2 from .resource_mapping import RESOURCE_MAPPING class FacebookClientAdapter(JSONAdapterMixin, TapiocaAdapter): api_root = 'https://graph.facebook.com/' resource_mapping = RESOURCE_MAPPING def get_request_kwargs(self, api_params, *args, **kwargs): params = super(FacebookClientAdapter, self).get_request_kwargs( api_params, *args, **kwargs) params['auth'] = OAuth2( api_params.get('client_id'), token={ 'access_token': api_params.get('access_token'), 'token_type': 'Bearer'}) return params def get_iterator_list(self, response_data): return response_data['data'] def get_iterator_next_request_kwargs(self, iterator_request_kwargs, response_data, response): paging = response_data.get('paging') if not paging: return url = paging.get('next') if url: return {'url': url} Facebook = generate_wrapper_from_adapter(FacebookClientAdapter) </code></pre>That's right, it's a entire client lib from a 37 line file plus the [ridiculously simple] <a href="https://github.com/vintasoftware/tapioca-facebook/blob/master/tapioca_facebook/resource_mapping.py">resource mapping</a>.You can benefit from the <a href="http://tapioca-wrapper.readthedocs.io/en/stable/flavours.html">flavours that were already developed</a>. Or you can create a new Tapioca for a public API or your private service using the <a href="https://github.com/vintasoftware/cookiecutter-tapioca">Cookiecutter template</a>.Resources:<ul><li><a href="https://github.com/vintasoftware/tapioca-wrapper">Project Github page</a></li><li><a href="http://tapioca-wrapper.readthedocs.io/en/stable/buildingawrapper.html">How to build a Tapioca wrapper</a></li><li><a href="http://tapioca-wrapper.readthedocs.io/en/stable/quickstart.html">Full project documentation</a></li></ul>Want read more about APIs? Check out these posts from our blog:<ul><li><a href="http://www.vinta.com.br/blog/2015/we-can-build-better-api-clients/">Seriously, we can build better web API clients</a></li><li><a href="http://www.vinta.com.br/blog/2015/cdrf/">Classy Django REST Framework Release</a></li></ul>