[BUG] It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.

Describe the bug

I would like to use the Spacy layer, but it's not working. The following code:

nlp = spacy.load('/opt/en_core_web_sm-2.2.5')

Raises the following error: It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory. I've listed what is inside the /opt directory, and there are only python dependencies within the python/ directory.

I have tried to use the layer for both Python37 and Python38.

Layer Version ARN:

arn:aws:lambda:us-east-1:113088814899:layer:Klayers-python37-spacy:18
arn:aws:lambda:us-east-1:770693421928:layer:Klayers-python38-spacy:11

Specific Layer ARN that didn't work (let's us know which version, region etc).

Framework:

Serverless Framework

[keithrozario]

This isn't a bug, but I've been getting these a lot.

So I had created a special layer just for the spacy model. It is the smallest model, but it'll get your function to work.

To use a spacy model, you'll need both the spacy layer and the spacy model layer.
arn:aws:lambda:us-east-1:113088814899:layer:Klayers-python37-spacy:18
arn:aws:lambda:us-east-1:770693421928:layer:Klayers-python38-spacy_model_en_small:1

Then from within the function, load the model using:
nlp = spacy.load("/opt/en_core_web_sm-2.2.5/en_core_web_sm/en_core_web_sm-2.2.5")

Hope that helps.

[keithrozario]

I just uploaded this today, so the layer arns for the model will only show up in next week's cycle, but you can just substitute the arn above for the region in which you're operating. Should work.

[himself65]

Response
{
  "errorMessage": "[E053] Could not read config.cfg from /opt/en_core_web_sm-2.2.5/en_core_web_sm/en_core_web_sm-2.2.5/config.cfg",
  "errorType": "OSError",
  "stackTrace": [
    "  File \"/var/lang/lib/python3.8/imp.py\", line 234, in load_module\n    return load_source(name, filename, file)\n",
    "  File \"/var/lang/lib/python3.8/imp.py\", line 171, in load_source\n    module = _load(spec)\n",
    "  File \"<frozen importlib._bootstrap>\", line 702, in _load\n",
    "  File \"<frozen importlib._bootstrap>\", line 671, in _load_unlocked\n",
    "  File \"<frozen importlib._bootstrap_external>\", line 843, in exec_module\n",
    "  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n",
    "  File \"/var/task/lambda_function.py\", line 10, in <module>\n    nlp = spacy.load(\"/opt/en_core_web_sm-2.2.5/en_core_web_sm/en_core_web_sm-2.2.5\")\n",
    "  File \"/opt/python/spacy/__init__.py\", line 50, in load\n    return util.load_model(\n",
    "  File \"/opt/python/spacy/util.py\", line 326, in load_model\n    return load_model_from_path(Path(name), **kwargs)\n",
    "  File \"/opt/python/spacy/util.py\", line 390, in load_model_from_path\n    config = load_config(config_path, overrides=dict_to_dot(config))\n",
    "  File \"/opt/python/spacy/util.py\", line 547, in load_config\n    raise IOError(Errors.E053.format(path=config_path, name=\"config.cfg\"))\n"
  ]
}

[keithrozario]

which version of python are you using?

[KabilanV]

I get this error when I tried running KLayers - SpaCy package on Lambda.
Any suggestions on how to solve this?

The following is what I get when I run the function.
Am I missing the version?

Response
{
  "errorMessage": "[E050] Can't find model '/opt/en_core_web_sm-2.25/en_core_web_sm/en_core_web_sm-2.25'. It doesn't seem to be a Python package or a valid path to a data directory.",
  "errorType": "OSError",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 7, in lambda_handler\n    nlp = spacy.load(\"/opt/en_core_web_sm-2.25/en_core_web_sm/en_core_web_sm-2.25\")\n",
    "  File \"/opt/python/spacy/__init__.py\", line 50, in load\n    return util.load_model(\n",
    "  File \"/opt/python/spacy/util.py\", line 331, in load_model\n    raise IOError(Errors.E050.format(name=name))\n"
  ]
}

Function Logs
START RequestId: 963dcec0-2536-4400-8208-691025f4a608 Version: $LATEST
[ERROR] OSError: [E050] Can't find model '/opt/en_core_web_sm-2.25/en_core_web_sm/en_core_web_sm-2.25'. It doesn't seem to be a Python package or a valid path to a data directory.
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 7, in lambda_handler
    nlp = spacy.load("/opt/en_core_web_sm-2.25/en_core_web_sm/en_core_web_sm-2.25")
  File "/opt/python/spacy/__init__.py", line 50, in load
    return util.load_model(
  File "/opt/python/spacy/util.py", line 331, in load_model
    raise IOError(Errors.E050.format(name=name))END RequestId: 963dcec0-2536-4400-8208-691025f4a608

[eritain]

@KabilanV , the version was mismatched for a while (2.2.5 versus 2.25), but you can find out exactly what your layers provide by temporarily changing your Lambda code to

# list what the layers loaded: contents of /opt
import os
def lambda_handler(event, context):
    loaded = [os.path.join(dp, f) for dp, dn, fn in os.walk("/opt") for f in fn]
    return "\n".join(loaded)

However, you will probably join me and @himself65 in a bigger problem, because I have the same problem even after everything is up to 2.2.5

There is a break in compatibility between some 2.x versions of en_core_web_sm and Spacy 3.x, I think that's what's going on.