DepCCGParser not working

[mspronesti]

Hello,
I'm opening this as a discussion and not as an issue as I'm not really sure this is lambeq's fault.
After installing lambeq like this

pip install lambeq[depccg]

and after installing depccg with elmo , rabank and elmo_rebank

pip install depccg
depccg_en download elmo
depccg_en download rebank
depccg_en download elmo_rebank

I tried to instanciate a DepCCGParser (e.g. for elmo)

parser = DepCCGParser(model='elmo')

but it throws the following exception

In [1]: from lambeq import DepCCGParser

In [2]: parser = DepCCGParser(model='elmo')
2022-10-10 11:50:49,281 - INFO - allennlp.models.archival - loading archive file /home/mpronesti/anaconda3/lib/python3.9/site-packages/depccg/models/lstm_parser_elmo.tar.gz from cache at /home/mpronesti/anaconda3/lib/python3.9/site-packages/depccg/models/lstm_parser_elmo.tar.gz
2022-10-10 11:50:49,281 - INFO - allennlp.models.archival - extracting archive file /home/mpronesti/anaconda3/lib/python3.9/site-packages/depccg/models/lstm_parser_elmo.tar.gz to temp dir /tmp/tmpzfylzhf0
---------------------------------------------------------------------------
BadGzipFile                               Traceback (most recent call last)
File ~/anaconda3/lib/python3.9/tarfile.py:1691, in TarFile.gzopen(cls, name, mode, fileobj, compresslevel, **kwargs)
   1690 try:
-> 1691     t = cls.taropen(name, mode, fileobj, **kwargs)
   1692 except OSError:

File ~/anaconda3/lib/python3.9/tarfile.py:1668, in TarFile.taropen(cls, name, mode, fileobj, **kwargs)
   1667     raise ValueError("mode must be 'r', 'a', 'w' or 'x'")
-> 1668 return cls(name, mode, fileobj, **kwargs)

File ~/anaconda3/lib/python3.9/tarfile.py:1531, in TarFile.__init__(self, name, mode, fileobj, format, tarinfo, dereference, ignore_zeros, encoding, errors, pax_headers, debug, errorlevel, copybufsize)
   1530     self.firstmember = None
-> 1531     self.firstmember = self.next()
   1533 if self.mode == "a":
   1534     # Move to the end of the archive,
   1535     # before the first empty block.

File ~/anaconda3/lib/python3.9/tarfile.py:2363, in TarFile.next(self)
   2362     else:
-> 2363         raise e
   2364 except ImportError:

File ~/anaconda3/lib/python3.9/tarfile.py:2336, in TarFile.next(self)
   2335 try:
-> 2336     tarinfo = self.tarinfo.fromtarfile(self)
   2337 except EOFHeaderError as e:

File ~/anaconda3/lib/python3.9/tarfile.py:1121, in TarInfo.fromtarfile(cls, tarfile)
   1118 """Return the next TarInfo object from TarFile object
   1119    tarfile.
   1120 """
-> 1121 buf = tarfile.fileobj.read(BLOCKSIZE)
   1122 obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors)

File ~/anaconda3/lib/python3.9/gzip.py:300, in GzipFile.read(self, size)
    299     raise OSError(errno.EBADF, "read() on write-only GzipFile object")
--> 300 return self._buffer.read(size)

File ~/anaconda3/lib/python3.9/_compression.py:68, in DecompressReader.readinto(self, b)
     67 with memoryview(b) as view, view.cast("B") as byte_view:
---> 68     data = self.read(len(byte_view))
     69     byte_view[:len(data)] = data

File ~/anaconda3/lib/python3.9/gzip.py:487, in _GzipReader.read(self, size)
    486 self._init_read()
--> 487 if not self._read_gzip_header():
    488     self._size = self._pos

File ~/anaconda3/lib/python3.9/gzip.py:435, in _GzipReader._read_gzip_header(self)
    434 if magic != b'\037\213':
--> 435     raise BadGzipFile('Not a gzipped file (%r)' % magic)
    437 (method, flag,
    438  self._last_mtime) = struct.unpack("<BBIxx", self._read_exact(8))

BadGzipFile: Not a gzipped file (b'<!')

During handling of the above exception, another exception occurred:

ReadError                                 Traceback (most recent call last)
Input In [2], in <cell line: 1>()
----> 1 parser = DepCCGParser(model='elmo')

File ~/anaconda3/lib/python3.9/site-packages/lambeq/text2diagram/depccg_parser.py:133, in DepCCGParser.__init__(self, lang, model, use_model_unary_rules, annotator, device, root_cats, verbose, **kwargs)
    131 depccg.lang.set_global_language_to(lang)
    132 self.annotator_fun = english_annotator.get(annotator, annotate_XX)
--> 133 self.supertagger, config = depccg.instance_models.load_model(model,
    134                                                              device)
    135 (self.apply_binary_rules,
    136  self.apply_unary_rules,
    137  self.category_dict,
    138  _) = depccg.allennlp.utils.read_params(config.config)
    140 if not use_model_unary_rules:

File ~/anaconda3/lib/python3.9/site-packages/depccg/instance_models.py:138, in load_model(variant, device)
    136 model_path, config = load_model_directory(variant)
    137 if config.framework == 'allennlp':
--> 138     supertagger = load_allennlp_tagger(model_path, device)
    139 elif config.framework == 'chainer':
    140     supertagger = load_chainer_tagger(model_path, device)

File ~/anaconda3/lib/python3.9/site-packages/depccg/allennlp/supertagger.py:62, in load_allennlp_tagger(model_path, device)
     60 if device >= 0:
     61     logger.info(f'sending the supertagger to gpu: {device}')
---> 62 archive = load_archive(model_path, cuda_device=device)
     63 predictor = SupertaggerPredictor.from_archive(
     64     archive, 'supertagger-predictor'
     65 )
     66 return AllennlpSupertagger(predictor)

File ~/anaconda3/lib/python3.9/site-packages/allennlp/models/archival.py:220, in load_archive(archive_file, cuda_device, overrides, weights_file)
    218     serialization_dir = resolved_archive_file
    219 else:
--> 220     with extracted_archive(resolved_archive_file, cleanup=False) as tempdir:
    221         serialization_dir = tempdir
    223 if weights_file:

File ~/anaconda3/lib/python3.9/contextlib.py:119, in _GeneratorContextManager.__enter__(self)
    117 del self.args, self.kwds, self.func
    118 try:
--> 119     return next(self.gen)
    120 except StopIteration:
    121     raise RuntimeError("generator didn't yield") from None

File ~/anaconda3/lib/python3.9/site-packages/allennlp/models/archival.py:301, in extracted_archive(resolved_archive_file, cleanup)
    299 tempdir = tempfile.mkdtemp()
    300 logger.info(f"extracting archive file {resolved_archive_file} to temp dir {tempdir}")
--> 301 with tarfile.open(resolved_archive_file, "r:gz") as archive:
    302     archive.extractall(tempdir)
    303 yield tempdir

File ~/anaconda3/lib/python3.9/tarfile.py:1638, in TarFile.open(cls, name, mode, fileobj, bufsize, **kwargs)
   1636     else:
   1637         raise CompressionError("unknown compression type %r" % comptype)
-> 1638     return func(name, filemode, fileobj, **kwargs)
   1640 elif "|" in mode:
   1641     filemode, comptype = mode.split("|", 1)

File ~/anaconda3/lib/python3.9/tarfile.py:1695, in TarFile.gzopen(cls, name, mode, fileobj, compresslevel, **kwargs)
   1693     fileobj.close()
   1694     if mode == 'r':
-> 1695         raise ReadError("not a gzip file")
   1696     raise
   1697 except:

ReadError: not a gzip file

which is by the way the same kind of error I get if I try to run depcgg_en download (i.e. installing the default model. This is, I think, depccg's fault as they also have issues pending on the repository).

Any workaround to make DepCCGParser work ? Does it work for you ?
Another thing I don't understand is how come that this fails to work while WebParser (which uses depccg by default, right ?) works.
Thanks :)

[ianyfan]

Yes, unfortunately we also run into the same problems, so it is an issue on depccg's side. We've just been using the base model, which seems to work.

[mspronesti]

Hi there,
thanks for your answer. Does the base model work for you without manually untarring the archive ? The follows does not work for me (raises the exception mentioned in this issue)

pip install lambeq[depccg]
pip install depccg
depccg_en download 

Downloading the archive and untarring it in the correct place does the job.

[ianyfan]

Yes, that is also the case for us, which is why we wrote a helper script https://github.com/CQCL/lambeq/blob/main/contrib/download_depccg_model.py specifically for downloading the depccg model. (though it is a bit old so hopefully it still works)