Make schema validation optional #100

1vn · 2019-05-07T21:58:52Z

Checklist:

Run make test and make lint
Test end to end manually (e.g. build/push all images, restart local operator, and run cx refresh in an example folder)
Update documentation
Update examples and cx init
Alert team if dev environment changed
Cherry-pick into release branches if it's a bugfix
Delete the branch once it's merged

aggregates go + python yes it works propagate metadata

…c-optional

pkg/lib/strings/operations.go

pkg/operator/context/aggregators.go

pkg/operator/api/context/aggregators.go

pkg/operator/api/userconfig/aggregates.go

pkg/operator/api/userconfig/config.go

pkg/operator/api/userconfig/transformed_columns.go

pkg/workloads/spark_job/spark_util.py

pkg/workloads/lib/tf_lib.py

pkg/workloads/spark_job/spark_util.py

pkg/workloads/tf_api/api.py

pkg/workloads/lib/context.py

pkg/operator/api/userconfig/config.go

pkg/operator/api/userconfig/errors.go

pkg/workloads/lib/context.py

pkg/workloads/spark_job/spark_util.py

1vn added 13 commits May 5, 2019 10:38

ResourceConfigField -> ResourceField

214e1cc

go transformer changes

719ed13

aggregates go + python yes it works propagate metadata

Merge branch 'master' into spec-optional

b26eccc

comment out error stack printing

5c835c2

format

b54ffd3

simplify fetch_metadata

162bc86

clean up

b8db138

fix transform spark bug and training_columns

a35f089

format

fbf9f56

fix url

78e0095

unfix url

09992cc

Merge branch 'spec-optional' of github.com:cortexlabs/cortex into spe…

aa9e900

…c-optional

merge master

0f6954d

deliahu reviewed May 9, 2019

View reviewed changes

pkg/lib/strings/operations.go Outdated Show resolved Hide resolved

pkg/operator/context/aggregators.go Show resolved Hide resolved

pkg/operator/context/aggregators.go Show resolved Hide resolved

1vn requested review from ospillinger and removed request for ospillinger May 10, 2019 15:23

1vn added 2 commits May 10, 2019 17:39

address some comments

36c5fcf

check 5 samples

6a3a9bd

deliahu reviewed May 13, 2019

View reviewed changes

pkg/workloads/spark_job/spark_util.py Outdated Show resolved Hide resolved

1vn added 10 commits May 13, 2019 13:29

remove skip validation

c5f760d

remove outdated check

0bd6b73

add AlphaNumericDashDotUnderscoreEmpty

081b9b3

address some comments - lowercase resourceFields and replace error

7fedb6f

cache anon specs

9445688

fix autogen

14e45ff

progress

03a22bc

lazy load metadata

c4a04bb

clean up spark

cd38e52

format and lint

909a001

vishalbollu reviewed May 17, 2019

View reviewed changes

pkg/workloads/lib/tf_lib.py Outdated Show resolved Hide resolved

vishalbollu reviewed May 17, 2019

View reviewed changes

pkg/workloads/spark_job/spark_util.py Outdated Show resolved Hide resolved

remove more camel case

076c020

vishalbollu approved these changes May 17, 2019

View reviewed changes

fix more camel case

486c026

deliahu reviewed May 17, 2019

View reviewed changes

address some comments

79657c4

deliahu reviewed May 17, 2019

View reviewed changes

pkg/workloads/lib/context.py Outdated Show resolved Hide resolved

1vn added 2 commits May 18, 2019 17:24

refactor metadata

471e98f

use raw_dataset key

2d5602c

deliahu reviewed May 19, 2019

View reviewed changes

1vn added 5 commits May 21, 2019 11:49

address some comments

8c7fa9b

move type inference to validate_trannsformer

e493684

validate inferred types from transform spark and python

c1551c0

pass type downstream

8abc9dc

add comment about transform_python

b36218e

deliahu reviewed May 21, 2019

View reviewed changes

pkg/workloads/spark_job/spark_util.py Outdated Show resolved Hide resolved

1vn added 3 commits May 21, 2019 15:29

remove unused

5a7bdd3

Merge branch 'master' into spec-optional

7583e39

address comments

2aa5a99

deliahu reviewed May 21, 2019

View reviewed changes

pkg/workloads/spark_job/spark_util.py Outdated Show resolved Hide resolved

format

95d98e1

deliahu reviewed May 21, 2019

View reviewed changes

pkg/workloads/spark_job/spark_util.py Outdated Show resolved Hide resolved

wrap more code in try

694b08d

deliahu reviewed May 21, 2019

View reviewed changes

pkg/workloads/spark_job/spark_util.py Outdated Show resolved Hide resolved

1vn added 2 commits May 21, 2019 17:43

move type check before value check

5b5f0b6

address comments

b2fd9d7

deliahu approved these changes May 21, 2019

View reviewed changes

1vn merged commit 48d9700 into master May 21, 2019

1vn deleted the spec-optional branch May 21, 2019 22:37

Make schema validation optional #100

Make schema validation optional #100

Uh oh!

Conversation

1vn commented May 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

1vn commented May 7, 2019 •

edited

Loading