Skip to content

Commit 97f6d4e

Browse files
wprichardwprichard
authored
Merge pull request #74 from oracle/piday
Piday
2 parents 04b2000 + fe13c1f commit 97f6d4e

File tree

220 files changed

+1289
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

220 files changed

+1289
-0
lines changed

‎labs/PiDay-2022-Vision/Lab-0-intro.md

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Pi Day Hands-On Lab
2+
# Be an AI Wizard: No ML magic required
3+
4+
## Lab 0: Introduction
5+
6+
This lab is built for use as a workshop at the Oracle Pi Day event in March 2022. It uses Oracle Cloud Infrastructure (OCI) Vision service to show how AI services can be used with applications. The workshop guides the user to run a safety application that that looks for persons not wearing hardhats in images of industrial scenes.
7+
8+
OCI Vision is a serverless, multi-tenant cloud service, accessible using the Console, over REST APIs, or via programming language SDKs. It detects and classifies objects in images. Batches of images can be processed using asynchronous API endpoints. OCI Vision features include both document AI for document-centric images and image analysis for object and scene-based images.
9+
10+
### 1. Image Analysis
11+
12+
* Object Detection is an image analysis feature that detects and locates the objects in an image. For example, if the image is of a living room, Vision locates the objects therein, such as a chair, a sofa, and a TV. It then draws bounding boxes around the objects and identifies them. It can also be used for visual anomaly detection by training it to detect anomalous objects in images.
13+
* Image Classification is an image analysis feature used to identify scene-based features and objects in an image.
14+
* Text Recognition (also known as Optical Character Recognition) means Vision can detect and recognize text in an image. Vision draws bounding boxes around the printed or hand-written text it locates in an image, and digitizes the text.
15+
16+
### 2. Document AI
17+
18+
* Text Recognition (also known as Optical Character Recognition) means Vision can detect and recognize text in a document. Vision draws bounding boxes around the printed or hand-written text it locates in an image, and digitizes the text.
19+
* Document Classification determines the type of document, such as a tax form, an invoice, or a receipt.
20+
* Language Classification detects the language of document based on visual features.
21+
* Table Extraction extracts content in tabular format, maintaining row/column relationships of cells.
22+
* Key Value Extraction (Receipts) can be used to identify values for predefined keys in a receipt. For example, if a receipt includes a merchant name, merchant address, or merchant phone number, Vision can identify these values and return them as a key value pair.
23+
24+
Document AI is a key building block for scenarios like business process automation (also called RPA for Robotic Process Automation), automated receipt processing, semantic search, and the automatic extraction of information from unstructured content like scanned documents.
25+
26+
Custom model training tailors base models (through transfer learning approaches) to make deep learning models tuned to the training data. Model selection, resource planning, and deployment are all managed by Vision.
27+
28+
*Estimated Workshop Time*: 90 minutes
29+
30+
### Objectives
31+
32+
In this workshop, you will:
33+
34+
* Configure an OCI tenancy for OCI Vision
35+
* Get familiar with the OCI Console and exercise OCI Vision features with it
36+
* Learn how to use REST API to communicate with OCI Vision endpoints
37+
* Execute a sample application that processes a batch of images using object detection
38+
* Learn how to train an image classification or object detection model through the OCI console
39+
40+
### Prerequisites
41+
- None
42+
- Familiar with Python programming is recommended.
43+
44+
[Proceed to the next lab](./Lab-1-tenancy-access.md).

‎labs/PiDay-2022-Vision/Lab-1-tenancy-access.md

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Lab 1: Setup a tenancy for OCI Vision
2+
## Introduction
3+
To do the lab, you will need an Oracle Cloud account. This procedure will guide you through the steps of getting an Oracle Cloud Free Tier account and signing in. Then you will prepare the new OCI tenancy to use OCI Vision.
4+
5+
### Existing Cloud Accounts
6+
If you already have access to an Oracle Cloud account, including an Oracle Cloud account using [Oracle Universal Credits](https://docs.oracle.com/en/cloud/get-started/subscriptions-cloud/csgsg/universal-credits.html), skip to **Task 2** to sign in to your cloud tenancy.
7+
8+
### Objectives
9+
* Sign-up for a free Oracle cloud account
10+
* Sign-in to your Oracle cloud account
11+
12+
### Prerequisites
13+
* A valid email address (the same one you used for the event registration)
14+
* Ability to receive SMS text verification (only if your email isn't recognized)
15+
16+
## **TASK 1:** Create Your Free Trial Account
17+
> **Note**: Interfaces in the following screenshots might look different from the interfaces you will see.
18+
19+
If you already have a cloud account with unused cloud credits, skip to **Task 2**.
20+
21+
1. Open up a web browser to access the Oracle Cloud account registration form at [signup.cloud.oracle.com](https://signup.cloud.oracle.com).
22+
23+
You will be presented with a registration page.
24+
![](./images/cloud-infrastructure.png " ")
25+
1. Enter the following information to create your Oracle Cloud Free Tier account.
26+
* Choose your **Country**
27+
* Enter your **Name** and **Email**.
28+
* Check the box to indicate you are human and complete the captcha
29+
30+
1. Once you have entered a valid email address, select the **Verify my email** button.
31+
The screen will appear as follows after you select the button:
32+
![](images/verify-email.png " ")
33+
34+
1. As a participant in this event, you are being offered an Oracle Cloud Trial with $500 in free cloud credits for 30 days. You will see a popup for this offer. Click the **Select Offer** button.
35+
![](images/special-offer.png)
36+
37+
1. Go to your email. You will see an account validation email from Oracle in your inbox. The email will be similar to the following:
38+
![](images/verification-mail.png " ")
39+
40+
1. Select the link (if possible) or copy and paste the link into your browser.
41+
42+
1. Enter the following information to create your Oracle Cloud Free Tier account.
43+
- Choose a **Password**. Remember what you entered. You'll need this later to sign in.
44+
- Enter your **Company Name**
45+
- Your **Cloud Account Name** will be generated automatically based on your inputs, but you can change that name by entering a new value. Remember what you entered. You'll need this name later to sign in.
46+
- Choose a **Home Region**. Your Home Region cannot be changed once you sign-up. Select *US East (Ashburn)*
47+
- Click **Continue**
48+
![](images/account-info.png " ")
49+
50+
1. Enter your Address information. Click **Continue**.
51+
![](images/free-tier-address.png " ")
52+
53+
1. Check the box under Agreement to start your free trial.
54+
![](images/free-tier-agreement.png " ")
55+
56+
> **Note** - if you selected the offer for the free Cloud Trial above, you will not be asked for a payment verification method.
57+
58+
1. You will have to wait briefly while your account is setup.
59+
60+
![](images/setup-wait.png " ")
61+
62+
1. Then you will be redirected to the OCI Console. We'll use the console in the next task.
63+
64+
1. You'll also receive two emails from Oracle. One email will be the initial notification that provisioning is underway and it will list **important information** you will need in this workshop:
65+
- Cloud account
66+
- Username
67+
- Link (button) to access your account
68+
69+
70+
The other email will be a notification that provisioning is complete. Here is a copy of the final notification:
71+
![](images/account-provisioned.png " ")
72+
73+
## **TASK 2:** Sign in to Your Account
74+
When you sign up for an Oracle Cloud account, a user is created for you in Oracle Identity Cloud Service with the username and password you selected at sign up. You can use this single sign-on option to sign in to Oracle Cloud Infrastructure and then navigate to other Oracle Cloud services without reauthenticating. This user has administrator privileges for all the Oracle Cloud services included with your account.
75+
76+
1. Use the **account access link in your confirmation email** (this provides a shortcut to the sign-in for your account and region). Enter your user name and password and click **Sign In**. Your username is your email address. The password is what you chose when you signed up for an account.
77+
78+
![](images/direct-sign-in.png)
79+
80+
Alternatively, go to [cloud.oracle.com](https://cloud.oracle.com) . Enter your **Cloud Account Name** and click **Next**. This is the name you chose while creating your account in the previous section. It's NOT your email address. If you've forgotten the name, see the confirmation email.
81+
![](images/cloud-oracle.png " ")
82+
- Click **Continue** to sign in using the *oraclecloudidentityservice*.
83+
84+
![](images/cloud-login-tenant-single-sigon.png " ")
85+
86+
- Enter your Cloud Account credentials and click **Sign In**. Your username is your email address. The password is what you chose when you signed up for an account.
87+
88+
![](images/oci-signin-single-signon.png " ")
89+
90+
1. You are now signed in to Oracle Cloud!
91+
92+
![](images/oci-console-home-page.png " ")
93+
94+
[Proceed to the next lab](./Lab-2-analyze-vision.md).

‎labs/PiDay-2022-Vision/Lab-2-analyze-vision.md

Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,180 @@
1+
# Lab 2: Use Vision Service through the OCI Console
2+
3+
## Introduction
4+
In this session, we will walk through the OCI Console to familiarize ourselves with the Vision Service. We'll discuss the data requirements and formats, and provide some sample datasets as examples. We will also show you how to upload to Oracle Object Storage for later to train a custom model.
5+
6+
### Objectives
7+
8+
In this lab, you will:
9+
- Setup the OCI environment either through **Stacks** (preferred) or through Manual Policy setup
10+
- Understand the data requirements and data formats for analyzing images.
11+
- Be able to download prepared sample datasets and upload the downloaded dataset into OCI (Oracle Cloud Infrastructure) object storage.
12+
- Get familiar with the OCI console and be able to demo key vision features with it.
13+
14+
### Prerequisites
15+
16+
* A trial or paid Oracle cloud account
17+
18+
## **Stack Setup** (preferred method)
19+
A stack is a blueprint for a collection of cloud resources that can be automatically provisioned using Terraform in the OCI Console. OCI contains a stack template for OCI Vision that we will use to quickly configure the environment.
20+
21+
1. In the OCI Console, use the three-bar icon in the upper left corner of the page to open the menu. Go to **Developer Services** and under *Resource Manager* select **Stacks**.
22+
23+
![](images/Stacks-1.jpg)
24+
25+
2. Select compartment **\<cloud account> (root)**, then click **Create Stack**.
26+
27+
![](images/Stacks-2.jpg)
28+
29+
3. In the *Create Stack* dialog, select the **Template** radio button.
30+
31+
![](images/stackselecttemplate.png)
32+
33+
4. Click **Select Template**
34+
35+
5. In the *Browse Templates* dialog, on the *Service* tab, check **Vision**, then click **Select Template**.
36+
37+
![](images/Stacks-Select-Template-3.jpg)
38+
39+
6. Back in the *Create Stack* dialog, change the **Description** to *Provision Vision and its Prerequisites* or something descriptive for you.
40+
41+
![](images/stackdescription.png)
42+
43+
7. Ensure **Compartment** is set to the name of your **root** compartment
44+
45+
8. Click **Next**.
46+
47+
9. On the next _Configure Variables_ page, check **Use custom models** and **Use batch analysis** checkboxes and click **Next**
48+
49+
50+
51+
![](images/Stacks-Options-5.jpg)
52+
53+
10. On the Review page, check **Run Apply**, then click **Create**. The stack creation will run for a few minutes and then complete on its own. You can skip the **Manual Policy Setup** below and start **Task 1**
54+
55+
![](images/create-stack-review.png)
56+
57+
## **Manual Policy Setup** (not needed if stack setup steps are completed. Skip ahead to Task 1)
58+
59+
Before you start using OCI Vision, your tenancy administrator should set up the following policies by following below steps:
60+
61+
1. Navigate to Policies
62+
63+
Log into OCI Cloud Console. Using the Burger Menu on the top left corner, navigate to Identity & Security and click it, and then select Policies item under Identity.
64+
![](./images/policy1.png " ")
65+
66+
67+
1. Create Policy
68+
69+
Click Create Policy </br>
70+
![](./images/policy2.png " ")
71+
72+
73+
1. Create a new policy with the following statements:
74+
75+
If you want to allow all the users in your tenancy to use vision service, create a new policy with the below statement:
76+
```
77+
allow any-user to use ai-service-vision-family in tenancy
78+
```
79+
![](./images/policy3.png " ")
80+
81+
82+
If you want to limit access to a user group, create a new policy with the below statement: </br>
83+
```
84+
allow group <group-name> to use ai-service-vision-family in tenancy
85+
```
86+
![](./images/policy4.png " ")
87+
88+
## **Task 1:** Understand Data Requirements
89+
90+
The vision service works with multiple formats of image data in order to detect objects, assign labels to images, extract text, and more. The service accepts data through Object Storage and locally stored images (if using via OCI console).
91+
92+
The service offers sync and async APIs to analyze images, with data requirements for each detailed below:
93+
94+
| API | Description | Supported Input Format |
95+
| --- | --- | --- |
96+
| sync API (analyzeImage, analyzeDocument) | Analyzes individual images | * JPG, PNG, (PDF and Tiff for analyzeDocument)<br>* Up to 5 MB<br>* Single image input |
97+
| async API <br>/jobs/startImageAnalysisjob <br>/jobs/start | Analyze multiple images or multi-page PDFs | * JPG, PNG (PDF and Tiff for analyzeDocument)<br>* Up to 2000 images input<br>* Supports multi-page PDF |
98+
99+
## **Task 2:** Upload Data to Object Storage
100+
101+
This is the set of steps to follow to test OCI Vision with your own sample images.
102+
103+
1. Create an Object Storage Bucket (This step is optional in case the bucket is already created)
104+
105+
a. First, from the OCI Services hamburger menu on the left, click **Storage** and then **Buckets**.
106+
![](./images/cloud-storage-bucket.png " ")
107+
108+
b. Then, Select Compartment from the left dropdown menu. Choose the compartment matching your name or company name.
109+
![](./images/create-compartment.png " ")
110+
111+
c. Next click **Create Bucket**.
112+
113+
d. Next, fill out the dialog box:
114+
* Bucket Name: Provide a name called **pidaydemo** for convenience. If you do not name it _pidaydemo_, please make a note of your name and remember to refer to it wherever the subsequent labs refer to the _pidaydemo_ bucket <br/>
115+
* Storage Tier: STANDARD
116+
117+
e. Then click Create
118+
![](./images/bucket-pidaydemo.jpg " ")
119+
120+
</br>2. Upload image files into Storage Bucket. </br>
121+
> **Note:** It is important to follow these steps correctly to prevent problems with Lab 3 and Lab 4.
122+
123+
a. Download the set of images that you will use from the following two folders, and store it on your local computer in separate folders called **lab-3** and **lab-4** for convenience: [Lab 3](https://github.com/oracle/oci-data-science-ai-samples/tree/piday/labs/PiDay-2022-Vision/Sample-Images/Lab-3) and [Lab 4](https://github.com/oracle/oci-data-science-ai-samples/tree/piday/labs/PiDay-2022-Vision/Sample-Images/Lab-4). The images are in zip files so you will need to unzip to view the images.
124+
125+
b. Switch to OCI window and click the Bucket Name for the Bucket that you just created, whether it was **pidaydemo** or any custom name you gave it.
126+
127+
c. Bucket detail window should now be visible. Click the dropdown called **More Actions** then **Create New Folder**. Create a folder called **lab-3** and another one called **lab-4**
128+
![](./images/create-newfolder.png " ")
129+
130+
d. Click the **lab-3** folder name and then click Upload. Browse to the folder in your computer in which you downloaded the files for **lab-3**. Drag and drop all files into the bucket. Click on Upload to complete the upload process.
131+
![](./images/lab-3objectstoragefolder.png " ")
132+
133+
e. Repeat the process for all the files for the **lab-4** folder too.
134+
135+
f. In the same way that you created folders for **lab-3** and **lab-4**, create another folder called **output** that will be used in a later lab.
136+
137+
More details on Object storage can be found on this page. [Object Storage Upload Page](https://oracle.github.io/learning-library/oci-library/oci-hol/object-storage/workshops/freetier/index.html?lab=object-storage) to see how to upload.
138+
139+
## **Task 3:** Demo Vision Service using the OCI Console
140+
141+
### 1. Navigate to the Vision Page of OCI Console
142+
143+
![](./images/navigate-to-ai-vision-menu.png " ")
144+
145+
### 2. Use Document AI features
146+
147+
a. On the Vision page, select **Document AI** on the left navigation menu. Document AI has a pre-built example receipt image that extracts both Key Values and Raw Text that are displayed on the right. Alternatively, you can also provide a document or image from local storage or OCI object storage. This invokes analyzeDocument API after the image is provided.
148+
149+
![](./images/document-ai-features.png " ")
150+
151+
b. Features you can test out:
152+
153+
| Feature | Description | Details on Console |
154+
| --- | --- | --- |
155+
| OCR (Optical Character Recognition) | Locates and digitizes text information from images | Text will appear under the "raw text" header of the results pane of the console [\[Reference\]](./images/ocr.png) |
156+
| Document Image Classification | Classifies documents into different types based on their visual appearance, high-level features, and extracted keywords | Classification along with confidence score appears directly under "Results" pane [\[Reference\]](./images/dic.png) |
157+
| Language Classification | Classifies the language of document based on visual features | Classification along with confidence score appears under document classification in Results pane [\[Reference\]](./images/language-classification.png) |
158+
| Table Extraction | Extracts content in tabular format, maintaining row/column relationships of cells | Toggle to the Table tab to get table information [\[Reference\]](./images/table-extraction.png) |
159+
| Key Value Extraction (Receipts) | Extracts values for predefined fields in a receipt | Analyze a receipt and select key value [\[Reference\]](./images/key-value-receipt.png) |
160+
| Searchable PDF output | Embeds a transparent layer on top of document image in PDF format to make it searchable by keywords | You need to test on a PDF document to use this feature. When you've selected a PDF, the searchable PDF button will be clickable. Clicking on it will download an OCR PDF to your computer. [\[Reference\]](./images/searchable-pdf-output.png) |
161+
162+
### 3. Use Image Analysis Features
163+
164+
a. On the Vision page, select "Object Detection" on the left navigation menu and a default image will be processed. This invokes analyzeImage API after the image is provided. There is a second sample image of a person in a hardhat. The custom application in a later lab will analyze a batch of images similar to this.
165+
166+
![](./images/vision-objdet1.png " ")
167+
168+
![](./images/vision-objdet2.png " ")
169+
170+
b. Features you can test out. We recommend completing **Object Detection** below as part of **Lab 4** that follows, and then doing _Image Classification_ on your own.
171+
172+
| Feature | Description | Details on Console |
173+
| --- | --- | --- |
174+
| Image classification | Categorizes object(s) within an image | Select "Image Classification." Labels and confidence scores will appear under the Results pane. [\[Reference\]](./images/image-classification.png) |
175+
| Object detection | Locates and identifies objects within an image | Select "Object Detection." Objects, confidence score, and highlighted bounding box will all appear under the Results pane. Clicking on one of the labels on the results pane will also highlight where on the image that object was detected. [\[Reference\]](./images/object-detection.png) |
176+
| OCR (Optical Character Recognition) | Locates and digitizes text information from images | Text will appear under the "raw text" header of the results pane of the console [\[Reference\]](./images/ocr.png) |
177+
178+
Congratulations on completing this lab!
179+
180+
[Proceed to the next lab](./Lab-3-vision-rest.md).

0 commit comments

Comments
 (0)