Integrate ouroboros - Batteries Now Included

[PythonLinks]

The ReadMe.md says:

`
It does not include very many python modules as many of the core modules are written in C not python. The converted modules are:

builtins
marshal
math
time
sys
`

What you need is the Ourobours library.

A standalone, pure Python implementation of the Python Standard Library. This allows Python default libraries to be used portably on any implementation of the Python virtual machine---not requiring CPython.

And here is the list of Modules.
https://github.com/beeware/ouroboros

Files 1488
Lines Of Code 419872

That should move this project very far forward.

[ncw]

Yes that project looks very interesting indeed!

I did a quick experiment to see what top level modules gpython could import from ouroboros right now to get an idea what modules ourobouros is expecting from the host python.

~/go/src/github.com/go-python/gpython/ouroboros/ouroboros$ for i in *.py; do gpython $i > /tmp/z 2>&1 && echo "$i OK" || echo "$i FAILED: `tail -1 /tmp/z`"  ; done
abc.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
aifc.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_struct'"
antigravity.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'errno'"
argparse.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
ast.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_ast'"
asynchat.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'select'"
asyncore.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'select'"
base64.py FAILED: 2019/08/11 11:10:57 AttributeError: "'module' has no attribute 'maxsize'"
bdb.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'errno'"
binhex.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_io'"
bisect.py OK
_bootlocale.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_locale'"
__builtins__.py OK
bz2.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_io'"
calendar.py FAILED: 2019/08/11 11:10:57 AttributeError: "'list' has no attribute 'append'"
cgi.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_io'"
cgitb.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_ast'"
chunk.py OK
cmath.py OK
cmd.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_string'"
_codecs.py OK
codecs.py FAILED: 2019/08/11 11:10:57 AttributeError: "'module' has no attribute 'byteorder'"
codeop.py OK
code.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'errno'"
_collections_abc.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
colorsys.py OK
_compat_pickle.py FAILED: 2019/08/11 11:10:57 KeyError: 'FIXME can only have string keys!: [__builtin__ xrange]'
compileall.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'errno'"
configparser.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
contextlib.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
copy.py FAILED: 2019/08/11 11:10:57 AttributeError: "'type' has no attribute '__dict__'"
copyreg.py FAILED: 2019/08/11 11:10:57 NameError: "name 'callable' is not defined"
cProfile.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_lsprof'"
crypt.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_crypt'"
_csv.py FAILED: 2019/08/11 11:10:57 NameError: "name 'property' is not defined"
csv.py FAILED: 2019/08/11 11:10:57 AttributeError: "'module' has no attribute 'maxsize'"
datetime.py FAILED: 2019/08/11 11:10:57 AttributeError: "'list' has no attribute 'append'"
decimal.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
difflib.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
dis.py FAILED: 2019/08/11 11:10:57 AttributeError: "'type' has no attribute '__dict__'"
doctest.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
dummy_threading.py FAILED: 2019/08/11 11:10:57 ImportError: 'cannot import name modules'
_dummy_thread.py OK
enum.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
filecmp.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'errno'"
fileinput.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'errno'"
fnmatch.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'errno'"
formatter.py FAILED: 2019/08/11 11:10:57 AttributeError: "'module' has no attribute 'warnoptions'"
fractions.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
ftplib.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'errno'"
functools.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
__future__.py OK
genericpath.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'errno'"
getopt.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'errno'"
getpass.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_weakref'"
gettext.py FAILED: 2019/08/11 11:10:57 AttributeError: "'module' has no attribute 'byteorder'"
glob.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'errno'"
gzip.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named '_struct'"
hashlib.py FAILED: 2019/08/11 11:10:57 AttributeError: "'dict' has no attribute 'get'"
heapq.py FAILED: 2019/08/11 11:10:57 AttributeError: "'list' has no attribute 'append'"
hmac.py FAILED: 2019/08/11 11:10:57 AttributeError: "'module' has no attribute 'warnoptions'"
imaplib.py FAILED: 2019/08/11 11:10:57 ImportError: "No module named 'binascii'"
imghdr.py FAILED: 2019/08/11 11:10:58 AttributeError: "'list' has no attribute 'append'"
imp.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_imp'"
__init__.py OK
inspect.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_ast'"
io.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_io'"
ipaddress.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_weakref'"
itertools.py OK
keyword.py FAILED: 2019/08/11 11:10:58 TypeError: "cannot create 'frozenset' instances"
linecache.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
locale.py FAILED: 2019/08/11 11:10:58 AttributeError: "'module' has no attribute 'byteorder'"
lzma.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_io'"
macpath.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
macurl2path.py FAILED: 2019/08/11 11:10:58 AttributeError: "'module' has no attribute 'maxsize'"
mailbox.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
mailcap.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
_markupbase.py FAILED: 2019/08/11 11:10:58 AttributeError: "'module' has no attribute 'maxsize'"
mimetypes.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
modulefinder.py FAILED: 2019/08/11 11:10:58 AttributeError: "'type' has no attribute '__dict__'"
netrc.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
nntplib.py FAILED: 2019/08/11 11:10:58 AttributeError: "'module' has no attribute 'maxsize'"
ntpath.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
nturl2path.py OK
numbers.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_weakref'"
opcode.py FAILED: 2019/08/11 11:10:58 AttributeError: "'list' has no attribute 'append'"
operator.py OK
optparse.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
os.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
_osx_support.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
pathlib.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
pdb.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
__phello__.foo.py OK
pickle.py FAILED: 2019/08/11 11:10:58 AttributeError: "'type' has no attribute '__dict__'"
pickletools.py FAILED: 2019/08/11 11:10:58 AttributeError: "'module' has no attribute 'byteorder'"
pipes.py FAILED: 2019/08/11 11:10:58 AttributeError: "'module' has no attribute 'maxsize'"
pkgutil.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_weakref'"
platform.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_weakref'"
plistlib.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'binascii'"
poplib.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
posixpath.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
pprint.py FAILED: 2019/08/11 11:10:58 AttributeError: "'module' has no attribute 'maxsize'"
profile.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
pstats.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
pty.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'select'"
pyclbr.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_io'"
py_compile.py FAILED: 2019/08/11 11:10:58 AttributeError: "'int' has no attribute 'to_bytes'"
pydoc.py FAILED: 2019/08/11 11:10:58 AttributeError: "'int' has no attribute 'to_bytes'"
_pyio.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
queue.py FAILED: 2019/08/11 11:10:58 ImportError: 'cannot import name modules'
quopri.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
random.py FAILED: 2019/08/11 11:10:58 AttributeError: "'module' has no attribute 'warnoptions'"
reprlib.py OK
re.py FAILED: 2019/08/11 11:10:58 AttributeError: "'module' has no attribute 'maxsize'"
rlcompleter.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'atexit'"
runpy.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_imp'"
sched.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_weakref'"
selectors.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_weakref'"
shelve.py FAILED: 2019/08/11 11:10:58 AttributeError: "'type' has no attribute '__dict__'"
shlex.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
shutil.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
_sitebuiltins.py OK
site.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
smtpd.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named 'errno'"
smtplib.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_socket'"
sndhdr.py FAILED: 2019/08/11 11:10:58 AttributeError: "'list' has no attribute 'append'"
socket.py FAILED: 2019/08/11 11:10:58 ImportError: "No module named '_socket'"
socketserver.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_socket'"
sre_compile.py FAILED: 2019/08/11 11:10:59 AttributeError: "'module' has no attribute 'maxsize'"
sre_constants.py FAILED: 2019/08/11 11:10:59 AttributeError: "'module' has no attribute 'maxsize'"
sre_parse.py FAILED: 2019/08/11 11:10:59 ImportError: 'cannot import name ATCODES'
_sre.py FAILED: 2019/08/11 11:10:59 ImportError: 'cannot import name ATCODES'
ssl.py FAILED: 2019/08/11 11:10:59 AttributeError: "'module' has no attribute 'maxsize'"
statistics.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_weakref'"
stat.py OK
stringprep.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named 'unicodedata'"
string.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_string'"
_strptime.py FAILED: 2019/08/11 11:10:59 AttributeError: "'module' has no attribute 'byteorder'"
struct.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_struct'"
subprocess.py FAILED: 2019/08/11 11:10:59 AttributeError: "'module' has no attribute 'platform'"
sunau.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_weakref'"
symbol.py FAILED: 2019/08/11 11:10:59 AttributeError: "'dict' has no attribute 'items'"
symtable.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_symtable'"
sysconfig.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named 'errno'"
tabnanny.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named 'errno'"
tarfile.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named 'errno'"
telnetlib.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_socket'"
tempfile.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_weakref'"
textwrap.py FAILED: 2019/08/11 11:10:59 AttributeError: "'module' has no attribute 'maxsize'"
this.py FAILED: 2019/08/11 11:10:59 AttributeError: "'str' has no attribute 'join'"
_threading_local.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_weakref'"
threading.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_thread'"
timeit.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named 'gc'"
tokenize.py FAILED: 2019/08/11 11:10:59 AttributeError: "'module' has no attribute 'byteorder'"
token.py FAILED: 2019/08/11 11:10:59 AttributeError: "'dict' has no attribute 'items'"
traceback.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named 'errno'"
tracemalloc.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_weakref'"
trace.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named 'errno'"
tty.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named 'termios'"
turtle.py FAILED: 2019/08/11 11:10:59 AttributeError: "'module' has no attribute 'platform'"
types.py FAILED: 2019/08/11 11:10:59 AttributeError: "'type' has no attribute '__dict__'"
uuid.py FAILED: 2019/08/11 11:10:59 NameError: "name 'property' is not defined"
uu.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named 'binascii'"
warnings.py FAILED: 2019/08/11 11:10:59 AttributeError: "'module' has no attribute 'warnoptions'"
wave.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named 'audioop'"
weakref.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_weakref'"
_weakrefset.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_weakref'"
webbrowser.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named 'errno'"
xdrlib.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_struct'"
zipfile.py FAILED: 2019/08/11 11:10:59 ImportError: "No module named '_io'"

So 17/170 however the failures are weighted heavily to just a small number of things

~/go/src/github.com/go-python/gpython/ouroboros/ouroboros$ for i in *.py; do gpython $i > /tmp/z 2>&1 && echo "$i OK" || echo "$i FAILED: `tail -1 /tmp/z`"  ; done | grep FAILED | cut -d' ' -f 5- | sort | uniq -c | sort -rn
     39 ImportError: "No module named 'errno'" # not too hard to make a minimal implementation
     25 ImportError: "No module named '_weakref'" #not sure we can do weakrefs in go, but we could dummy it up
     12 AttributeError: "'module' has no attribute 'maxsize'" # this is sys.maxsize
      7 ImportError: "No module named '_io'"
      6 AttributeError: "'type' has no attribute '__dict__'" # this probably needs fixing!
      6 AttributeError: "'module' has no attribute 'byteorder'"
      6 AttributeError: "'list' has no attribute 'append'" # this should be easy to fix
      4 ImportError: "No module named '_struct'"
      4 ImportError: "No module named '_socket'"
      4 AttributeError: "'module' has no attribute 'warnoptions'"
      3 ImportError: "No module named 'select'"
      3 ImportError: "No module named 'binascii'"
      3 ImportError: "No module named '_ast'"
      2 NameError: "name 'property' is not defined"
      2 ImportError: "No module named '_string'"
      2 ImportError: "No module named '_imp'"
      2 ImportError: 'cannot import name modules'
      2 ImportError: 'cannot import name ATCODES'
      2 AttributeError: "'module' has no attribute 'platform'"
      2 AttributeError: "'int' has no attribute 'to_bytes'"
      2 AttributeError: "'dict' has no attribute 'items'"
      1 TypeError: "cannot create 'frozenset' instances"
      1 NameError: "name 'callable' is not defined"
      1 KeyError: 'FIXME can only have string keys!: [__builtin__ xrange]'
      1 ImportError: "No module named 'unicodedata'"
      1 ImportError: "No module named '_thread'"
      1 ImportError: "No module named 'termios'"
      1 ImportError: "No module named '_symtable'"
      1 ImportError: "No module named '_lsprof'"
      1 ImportError: "No module named '_locale'"
      1 ImportError: "No module named 'gc'"
      1 ImportError: "No module named '_crypt'"
      1 ImportError: "No module named 'audioop'"
      1 ImportError: "No module named 'atexit'"
      1 AttributeError: "'str' has no attribute 'join'"
      1 AttributeError: "'dict' has no attribute 'get'"

So it looks like there would be some easy wins to get more of it working.

Anyone want to help with this? @corona10 @sbinet ?

We should probably target some important modules to get working. "re" for instance.

[corona10]

I can help you Let's take a look!
I have experience with similar task before at grumpy project

[ncw]

I can help you Let's take a look!
I have experience with similar task before at grumpy project

That would be fantastic! Having ouroboros as a target to make work should focus our mind on things to fix.

[drew-512]

Hi @PythonLinks I appreciate the intention of this thread and I am hoping you are still perhaps interested in working together on furthering this. Check out gpython's v0.1.0 adds around multi-context support and my upcoming worklist.

I see gpython to be a prime and potent choice for Go devs looking to embed a capable interpreter in a world drowning in embedded interpreters (that often do not serve non-professional developer types who are more oriented on getting their job done quickly and with as little learning-curve as possible).

Drew