Deprecate TensorFlow "legacy mode"

[laurenyu]

Is your feature request related to a problem? Please describe.
Confusion around how to use the TensorFlow classes, e.g. #1406, #1444.

Describe the solution you'd like
The plan is to make TensorFlow, TensorFlowModel, and TensorFlowPredictor all use the TF script mode/TFS images by default. (These are the images that currently correspond to TF 1.11-2.1.) This includes:

• Remove legacy-mode parameters from the TensorFlow constructor, and document how to pass them as hyperparameters instead.
• Remove outdated TensorBoard-related code.
• Add a warning if framework_version < 1.11.
• Deprecate script_mode and endpoint_type parameters so that there is only one "mode/type" going forward.
• Move sagemaker.tensorflow.serving code to sagemaker.tensorflow.model (as suggested above).
• Remove outdated serializers/deserializers and copied TFS source code.
• Revise documentation, including removing the warnings about the change in training script format from the TensorFlow (and MXNet) documentation. Ideally, it would be nice to remove usage of the term "script mode" as much as possible because that is the only mode for training with frameworks on SageMaker now.

Describe alternatives you've considered
If there is high demand for "legacy mode" to stay, we could also just flip the defaults, and unify the serving classes.

[quocdat32461997]

@laurenyu Thanks for opening this. Also, Will v2.0.0 be compatible with Tensorflow 2.0 and above? Currently, my model is in Tensorflow v2.0. To deploy it, I have to save it to .pb format and launch another kernel in Tensorflow v1.14 to deploy it.

[laurenyu]

@quocdat32461997 yes. there is also currently support for TF 2.0 and 2.1 - have you been encountering issues with it?

[quocdat32461997]

Kinda. All my codes in TF v2 and try to deploy using Tensorflow Serving Model but there is no TF Serving v2 container in SageMaker. Hence, I have to deploy it using TF Serving v1.15. This trick works but not yet fully testing.

[laurenyu]

@quocdat32461997 the sagemaker.tensorflow.serving.Model class supports TF 2.0+ (and uses the TF 2 images documented here). what error led you to needing to port things back to TF 1.15?

[quocdat32461997]

In AWS SageMaker, I installed Tensorflow 2.1.0 by conda.
So, when I try to deploy using Tensorflow.serving.Model by executing python3 deploy.py and get the following error:

Traceback (most recent call last):
  File "deploy.py", line 98, in <module>
    main()
  File "deploy.py", line 93, in main
    sagemaker_model = Model(model_data = model_data, role = role, entry_point = 'inference.py')
TypeError: __init__() got an unexpected keyword argument 'entry_point'.

However, I wont have that error in TF 1.15.

[laurenyu]

hmm interesting. sagemaker.tensorflow.serving.Model does explicitly have entry_point as a keyword argument: https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/tensorflow/serving.py#L139. can you double check that you didn't import sagemaker.model.Model by accident?

[quocdat32461997]

You are right. I accidentally import sagemaker.model.Model. Then, I encounter ModuleNotFoundError: No module named 'numpy' when designing inference.py as following:

def input_handler(data, context):
    """ Pre-process request input before it is sent to TensorFlow Serving REST API
    Args:
        data (obj): the request data, in format of dict or string
        context (Context): an object containing request and configuration details
    Returns:
        (dict): a JSON-serializable dict that contains request body and headers
    """
    if context.request_content_type == 'application/x-image':
        payload = data.read()
        encoded_image = base64.b64encode(payload).decode('utf-8')
        instance = [{"b64": encoded_image}]
        return json.dumps({"instances": instance})
    elif context.request_content_type == 'application/x-npy':
        payload = numpy.load(data)
        encoded_image = base64.b64encode(payload).decode('utf-8')
        instance = [{"b64": encoded_image}]
        return json.dumps({"instances": instance})
    else:
        _return_error(415, 'Unsupported content type "{}"'.format(context.request_content_type or 'Unknown'))

Based on #831, before processing, I have to manually install numpy. Is there any final fix to issue?

[laurenyu]

As I mentioned in #831 (comment) and #831 (comment), you can use a requirements.txt file to get numpy. Full docs: https://github.com/aws/sagemaker-tensorflow-serving-container/#prepost-processing