Skip to content
GitLab
Commit ec51ef26 authored by davebshow's avatar davebshow
Browse files

updated docs

parent 4a45a3c2
Hide whitespace changes
Inline Side-by-side
LICENSE
View file @ ec51ef26
The MIT License (MIT)
Copyright [2017] [David M. Brown]
Copyright (c) 2015 David Michael Brown
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
README.md
View file @ ec51ef26
# aiogremin is no longer maintained. Please use Goblin: https://github.com/ZEROFAIL/goblin
# [aiogremlin 3.2.4](https://pypi.python.org/pypi/aiogremlin/3.2.4)
[Official Documentation](http://aiogremlin.readthedocs.org/en/latest/)
# [aiogremlin 0.1.3](https://pypi.python.org/pypi/aiogremlin/0.0.11)
`aiogremlin` is a port of the official `Gremlin-Python` designed for integration with
event loop based asynchronous Python networking libraries, including `asyncio`,
`aiohttp`, `tornado`, and `curio`. It uses the `async/await` syntax introduced
in PEP 492, and is therefore Python 3.5+ only.
## [Official Documentation](http://aiogremlin.readthedocs.org/en/latest/)
`aiogremlin` tries to follow `Gremlin-Python` as closely as possible both in terms
of API and implementation. It is regularly rebased against the official Apache Git
repository, and will be released according to the TinkerPop release schedule.
`aiogremlin` is a **Python 3** driver for the the [Tinkerpop 3 Gremlin Server](http://tinkerpop.incubator.apache.org/docs/3.0.0.M9-incubating/#gremlin-server). This module is built on [Asyncio](https://docs.python.org/3/library/asyncio.html) and [aiohttp](http://aiohttp.readthedocs.org/en/v0.15.3/index.html) `aiogremlin` is currently in **alpha** mode, but all major functionality has test coverage.
## Getting started
Since Python 3.4 is not the default version on many systems, it's nice to create a virtualenv that uses Python 3.4 by default. Then use pip to install `aiogremlin`. Using virtualenvwrapper on Ubuntu 14.04:
```bash
$ mkvirtualenv -p /usr/bin/python3.4 aiogremlin
$ pip install aiogremlin
```
Fire up the Gremlin Server:
```bash
$ ./bin/gremlin-server.sh
```
The `GremlinClient` communicates asynchronously with the Gremlin Server using websockets. The majority of `GremlinClient` methods are an `asyncio.coroutine`, so you will also need to use `asyncio`:
```python
>>> import asyncio
>>> from aiogremlin import GremlinClient
```
The Gremlin Server responds with messages in chunks, `GremlinClient.submit` submits a script to the server, and returns a `GremlinResponse` object. This object provides the methods: `get` and the property `stream`. `get` collects all of the response messages and returns them as a Python list. `stream` returns an object of the type `GremlinResponseStream` that implements a method `read`. This allows you to read the response without loading all of the messages into memory.
Note that the GremlinClient constructor and the create_client function take [keyword only arguments](https://www.python.org/dev/peps/pep-3102/) only!
Note that this *NOT* an official Apache project component, it is a
*THIRD PARTY PACKAGE!*
## Getting Started
```python
>>> loop = asyncio.get_event_loop()
>>> gc = GremlinClient(url='ws://localhost:8182/', loop=loop) # Default url
import asyncio
from aiogremlin import DriverRemoteConnection, Graph
# Use get.
>>> @asyncio.coroutine
... def get(gc):
... resp = yield from gc.submit("x + x", bindings={"x": 4})
... result = yield from resp.get()
... return result
>>> result = loop.run_until_complete(get(gc))
>>> result
[Message(status_code=200, data=[8], message={}, metadata='')]
loop = asyncio.get_event_loop()
>>> resp = result[0]
>>> resp.status_code
200
>>> resp # Named tuple.
Message(status_code=200, data=[8], message={}, metadata='')
async def go(loop):
remote_connection = await DriverRemoteConnection.open(
'ws://localhost:8182/gremlin', 'g')
g = Graph().traversal().withRemote(remote_connection)
vertices = await g.V().toList()
return vertices
# Use stream.
>>> @asyncio.coroutine
... def stream(gc):
... resp = yield from gc.submit("x + x", bindings={"x": 1})
... while True:
... result = yield from resp.stream.read()
... if result is None:
... break
... print(result)
>>> loop.run_until_complete(stream(gc))
Message(status_code=200, data=[2], message={}, metadata='')
>>> loop.run_until_complete(gc.close()) # Explicitly close client!!!
>>> loop.close()
```
For convenience, `aiogremlin` also provides a method `execute`, which is equivalent to calling `yield from submit()` and then `yield from get()` in the same coroutine.
```python
>>> loop = asyncio.get_event_loop()
>>> gc = GremlinClient(loop=loop)
>>> execute = gc.execute("x + x", bindings={"x": 4})
>>> result = loop.run_until_complete(execute)
>>> result
[Message(status_code=200, data=[8], message={}, metadata='')]
>>> loop.run_until_complete(gc.close()) # Explicitly close client!!!
>>> loop.close()
vertices = loop.run_until_complete(go(loop))
print(vertices)
# [v[1], v[2], v[3], v[4], v[5], v[6]]
```
README.txt
View file @ ec51ef26
=========================================================
aiogremlin - Async Python 3 driver for TP3 Gremlin Server
=========================================================
**alpha**
=====================================================
aiogremlin - Async Python 3.5+ port of Gremlin-Python
=====================================================
`Official Documentation`_
......
aiogremlin/__init__.py
View file @ ec51ef26
from aiogremlin.driver.cluster import Cluster
from aiogremlin.remote.driver_remote_connection import DriverRemoteConnection
from aiogremlin.gremlin_python import statics
from aiogremlin.gremlin_python.process import strategies
from aiogremlin.gremlin_python.process.graph_traversal import __
from aiogremlin.gremlin_python.process.traversal import Binding
from aiogremlin.gremlin_python.structure.graph import Graph
__version__ = "3.2.4"
aiogremlin/driver/client.py
View file @ ec51ef26
......@@ -14,33 +14,29 @@ class Client:
:param aiogremlin.cluster.Cluster cluster: Cluster used by
client
:param asyncio.BaseEventLoop loop:
:param dict aliases: Optional mapping for aliases. Default is `None`
"""
def __init__(self, cluster, loop, *, aliases=None, processor=None,
op=None):
def __init__(self, cluster, loop, *, aliases=None):
self._cluster = cluster
self._loop = loop
if aliases is None:
aliases = {}
self._aliases = aliases
if processor is None:
processor = ''
self._processor = processor
if op is None:
op = 'eval'
self._op = op
@property
def aliases(self):
"""Read-only property"""
return self._aliases
@property
def message_serializer(self):
"""Read-only property"""
return self.cluster.config['message_serializer']
@property
def cluster(self):
"""
Readonly property.
Read-only property.
:returns: The instance of
:py:class:`Cluster<aiogremlin.driver.cluster.Cluster>` associated with
......@@ -60,6 +56,11 @@ class Client:
"""
**coroutine** Submit a script and bindings to the Gremlin Server.
:param message: Can be an instance of
`Message<aiogremlin.gremlin_python.request.RequestMessage>` or
`Bytecode<aiogremlin.gremlin_python.process.traversal.Bytecode>`
or a `str` representing a raw Gremlin script
:param dict bindings: Optional bindings used with raw Grelmin
:returns: :py:class:`ResultSet<aiogremlin.driver.resultset.ResultSet>`
object
"""
......
aiogremlin/driver/cluster.py
View file @ ec51ef26
......@@ -35,6 +35,8 @@ class Cluster:
level interface used by the :py:mod:`aiogremlin` module.
:param asyncio.BaseEventLoop loop:
:param dict aliases: Optional mapping for aliases. Default is `None`
:param config: Optional cluster configuration passed as kwargs or `dict`
"""
DEFAULT_CONFIG = {
......@@ -73,8 +75,10 @@ class Cluster:
specified in configuration.
:param asyncio.BaseEventLoop loop:
:param dict aliases: Optional mapping for aliases. Default is `None`
:param str configfile: Optional configuration file in .json or
.yml format
:param config: Optional cluster configuration passed as kwargs or `dict`
"""
cluster = cls(loop, aliases=aliases, **config)
if configfile:
......@@ -84,12 +88,13 @@ class Cluster:
@property
def hosts(self):
"""Read-only property"""
return self._hosts
@property
def config(self):
"""
Readonly property.
Read-only property.
:returns: `dict` containing the cluster configuration
"""
......@@ -100,7 +105,7 @@ class Cluster:
**coroutine** Get connection from next available host in a round robin
fashion.
:returns: :py:class:`Connection<aiogremlin.connection.Connection>`
:returns: :py:class:`Connection<aiogremlin.driver.connection.Connection>`
"""
if not self._hosts:
await self.establish_hosts()
......@@ -136,6 +141,11 @@ class Cluster:
raise exception.ConfigurationError('Unknown config file format')
def config_from_yaml(self, filename):
"""
Load configuration from from YAML file.
:param str filename: Path to the configuration file.
"""
with open(filename, 'r') as f:
config = yaml.load(f)
config = self._process_config_imports(config)
......@@ -162,6 +172,11 @@ class Cluster:
return config
def config_from_module(self, module):
"""
Load configuration from Python module.
:param str filename: Path to the configuration file.
"""
if isinstance(module, str):
module = importlib.import_module(module)
config = dict()
......@@ -175,7 +190,8 @@ class Cluster:
"""
**coroutine** Get a connected client. Main API method.
:returns: A connected instance of `Client<aiogremlin.client.Client>`
:returns: A connected instance of
`Client<aiogremlin.driver.client.Client>`
"""
aliases = aliases or self._aliases
if not self._hosts:
......
aiogremlin/driver/connection.py
View file @ ec51ef26
......@@ -28,15 +28,16 @@ class Connection:
:py:meth:`Connection.open<aiogremlin.connection.Connection.open>`.
:param str url: url for host Gremlin Server
:param aiohttp.ClientWebSocketResponse ws: open websocket connection
:param aiogremlin.gremlin_python.driver.transport.AbstractBaseTransport transport:
Transport implementation
:param aiogremlin.gremlin_python.driver.protocol.AbstractBaseProtocol protocol:
Protocol implementation
:param asyncio.BaseEventLoop loop:
:param aiohttp.ClientSession: Client session used to establish websocket
connections
:param float response_timeout: (optional) `None` by default
:param str username: Username for database auth
:param str password: Password for database auth
:param int max_inflight: Maximum number of unprocessed requests at any
one time on the connection
:param float response_timeout: (optional) `None` by default
"""
def __init__(self, url, transport, protocol, loop, username, password,
max_inflight, response_timeout, message_serializer, provider):
......@@ -73,6 +74,9 @@ class Connection:
:param str url: url for host Gremlin Server
:param asyncio.BaseEventLoop loop:
:param aiogremlin.gremlin_python.driver.protocol.AbstractBaseProtocol protocol:
Protocol implementation
:param func transport_factory: Factory function for transports
:param ssl.SSLContext ssl_context:
:param str username: Username for database auth
:param str password: Password for database auth
......@@ -80,6 +84,8 @@ class Connection:
:param int max_inflight: Maximum number of unprocessed requests at any
one time on the connection
:param float response_timeout: (optional) `None` by default
:param message_serializer: Message serializer implementation
:param provider: Graph provider object implementation
:returns: :py:class:`Connection<aiogremlin.connection.Connection>`
"""
......@@ -100,7 +106,7 @@ class Connection:
@property
def closed(self):
"""
Check if connection has been closed.
Read-only property. Check if connection has been closed.
:returns: `bool`
"""
......@@ -119,10 +125,7 @@ class Connection:
"""
Submit a script and bindings to the Gremlin Server
:param str processor: Gremlin Server processor argument
:param str op: Gremlin Server op argument
:param args: Keyword arguments for Gremlin Server. Depend on processor
and op.
:param `RequestMessage<aiogremlin.gremlin_python.driver.request.RequestMessage>` message:
:returns: :py:class:`ResultSet<aiogremlin.driver.resultset.ResultSet>`
object
"""
......@@ -163,4 +166,4 @@ class Connection:
async def __aexit__(self, exc_type, exc, tb):
await self.close()
self._conn = None
self._transport = None
aiogremlin/driver/protocol.py
View file @ ec51ef26
......@@ -24,7 +24,7 @@ Message = collections.namedtuple(
class GremlinServerWSProtocol(protocol.AbstractBaseProtocol):
"""Implemenation of the Gremlin Server Websocket protocol"""
def __init__(self, message_serializer, username='', password=''):
if isinstance(message_serializer, type):
message_serializer = message_serializer()
......
aiogremlin/driver/resultset.py
View file @ ec51ef26
......@@ -12,6 +12,7 @@ def error_handler(fn):
if msg.status_code not in [200, 206]:
self.close()
raise exception.GremlinServerError(
msg.status_code,
"{0}: {1}".format(msg.status_code, msg.message))
msg = msg.data
return msg
......@@ -38,7 +39,7 @@ class ResultSet:
def queue_result(self, result):
if result is None:
self.done.set()
self.close()
self._response_queue.put_nowait(result)
@property
......@@ -68,10 +69,8 @@ class ResultSet:
return msg
def close(self):
"""Close response stream by setting done flag to true."""
self.done.set()
self._loop = None
self._response_queue = None
@error_handler
async def one(self):
......@@ -79,7 +78,6 @@ class ResultSet:
if not self._response_queue.empty():
msg = self._response_queue.get_nowait()
elif self.done.is_set():
self.close()
msg = None
else:
try:
......
aiogremlin/exception.py
View file @ ec51ef26
......@@ -23,7 +23,11 @@ class ConfigurationError(Exception):
class GremlinServerError(Exception):
pass
def __init__(self, status_code, msg):
super().__init__(msg)
self.status_code = status_code
self.msg = msg
class ResponseTimeoutError(Exception):
......
aiogremlin/gremlin_python/__init__.py
View file @ ec51ef26
......@@ -16,5 +16,10 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
'''
'''THIS FILE HAS BEEN MODIFIED BY DAVID M. BROWN TO SUPPORT PEP 492'''
__author__ = 'Marko A. Rodriguez (http://markorodriguez.com)'
from aiogremlin.gremlin_python.statics import *
from aiogremlin.gremlin_python.process.graph_traversal import __
from aiogremlin.gremlin_python.process.strategies import *
from aiogremlin.gremlin_python.process.traversal import Binding
aiogremlin/gremlin_python/process/traversal.py
View file @ ec51ef26
......@@ -75,19 +75,19 @@ class Traversal(object):
self.last_traverser = None
return temp
async def next(self, amount=None):
if amount is None:
return await self.__anext__()
else:
count = 0
tempList = []
while count < amount:
count = count + 1
try:
temp = await self.__anext__()
except StopAsyncIteration:
return tempList
tempList.append(temp)
return tempList
if not amount:
try:
return await self.__anext__()
except StopAsyncIteration:
return
results = []
for i in range(amount):
try:
result = await self.__anext__()
except StopAsyncIteration:
return results
results.append(result)
return results
Barrier = Enum('Barrier', 'normSack')
......
aiogremlin/remote/driver_remote_connection.py
View file @ ec51ef26
......@@ -11,6 +11,15 @@ __author__ = 'David M. Brown (davebshow@gmail.com)'
class DriverRemoteConnection:
"""
Remote connection to a Gremlin Server. Do not instantiate directly,
instead use :py:meth:`DriverRemoteConnection.open` or
:py:meth:`DriverRemoteConnection.using`
:param aiogremlin.driver.client.Client client:
:param asyncio.BaseEventLoop loop:
:param aiogremlin.driver.cluster.Cluster cluster:
"""
def __init__(self, client, loop, *, cluster=None):
self._client = client
......@@ -26,15 +35,32 @@ class DriverRemoteConnection:
return self._cluster.config
@classmethod
async def using(cls, cluster, aliases=None, *, loop=None):
async def using(cls, cluster, aliases=None):
"""
Create a :py:class:`DriverRemoteConnection` using a specific
:py:class:`Cluster<aiogremlin.driver.cluster.Cluster>`
:param aiogremlin.driver.cluster.Cluster cluster:
:param dict aliases: Optional mapping for aliases. Default is `None`.
Also accepts `str` argument which will be assigned to `g`
"""
client = await cluster.connect(aliases=aliases)
if not loop:
loop = asyncio.get_event_loop()
loop = cluster._loop
return cls(client, loop)
@classmethod
async def open(cls, url=None, aliases=None, loop=None, *,
graphson_reader=None, graphson_writer=None, **config):
"""
:param str url: Optional url for host Gremlin Server
:param dict aliases: Optional mapping for aliases. Default is `None`.
Also accepts `str` argument which will be assigned to `g`
:param asyncio.BaseEventLoop loop:
:param graphson_reader: Custom graphson_reader
:param graphson_writer: Custom graphson_writer
:param config: Optional cluster configuration passed as kwargs or `dict`
"""
if url:
parsed_url = urlparse(url)
config.update({
......@@ -54,11 +80,24 @@ class DriverRemoteConnection:
return cls(client, loop, cluster=cluster)
async def close(self):
"""
Close underlying cluster if applicable. If created with
:py:meth:`DriverRemoteConnection.using`, cluster is NOT closed.
"""
if self._cluster:
await self._cluster.close()
async def submit(self, bytecode):
"""Submit bytecode to the Gremlin Server"""
result_set = await self._client.submit(bytecode)
side_effects = RemoteTraversalSideEffects(result_set.request_id,
self._client)
return RemoteTraversal(result_set, side_effects)
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
await self.close()
self._client = None
self._cluster = None
aiogremlin/remote/driver_remote_side_effects.py
View file @ ec51ef26
......@@ -18,6 +18,7 @@ class RemoteTraversalSideEffects(traversal.TraversalSideEffects):
return await self.get(key)
async def keys(self):
"""Get side effect keys associated with Traversal"""
if not self._closed:
message = request.RequestMessage(
'traversal', 'keys',
......@@ -29,6 +30,7 @@ class RemoteTraversalSideEffects(traversal.TraversalSideEffects):
return self._keys
async def get(self, key):
"""Get side effects associated with a specific key"""
if not self._side_effects.get(key):
if not self._closed:
results = await self._get(key)
......@@ -47,6 +49,7 @@ class RemoteTraversalSideEffects(traversal.TraversalSideEffects):
return await self._aggregate_results(result_set)
async def close(self):
"""Release side effects"""
if not self._closed:
message = request.RequestMessage(
'traversal', 'close',
......
changes.txt deleted 100644 → 0
View file @ 4a45a3c2
0.0.1 - 4/2015: Birth!
0.0.2 - 5/1/2015: Added an init_pool method and a create_client constructor.
0.0.3 - 5/2/2015: Using ujson for serialization.
0.0.4 - 5/12/2015: Added support for sessions.
0.0.5 - 5/13/2015: Using EofStream terminator technique to prepare for 3.0.0.M9
0.0.7 - 5/20/2015: Full integration with aiohttp. Added Session object for HTTP
connection pooling. Added a context manager.
0.0.8 - 5/20/2015: Fixed bug in client constructor logic. Tested different client
connection handling techniques.
docs/aiogremlin.driver.aiohttp.rst 0 → 100644
View file @ ec51ef26
aiogremlin.driver.aiohttp package
=================================
Submodules
----------
aiogremlin.driver.aiohttp.transport module
------------------------------------------
.. automodule:: aiogremlin.driver.aiohttp.transport
:members:
:undoc-members:
:show-inheritance:
docs/aiogremlin.driver.rst 0 → 100644
View file @ ec51ef26
aiogremlin.driver package
=========================
Subpackages
-----------
.. toctree::
aiogremlin.driver.aiohttp
Submodules
----------
aiogremlin.driver.client module
-------------------------------
.. automodule:: aiogremlin.driver.client
:members:
:undoc-members:
:show-inheritance:
aiogremlin.driver.cluster module
--------------------------------
.. automodule:: aiogremlin.driver.cluster
:members:
:undoc-members:
:show-inheritance:
aiogremlin.driver.connection module