This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2020-07-02 02:31
Elapsed1h9m
Revisionbe13a819f66b34995b97757648b89fe0e732fa43
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/d0a7c373-d074-4895-b387-7b0e154ce7e8/targets/test'}}
resultstorehttps://source.cloud.google.com/results/invocations/d0a7c373-d074-4895-b387-7b0e154ce7e8/targets/test

No Test Failures!


Error lines from build-log.txt

... skipping 896 lines ...
NAME              LOCATION    MASTER_VERSION  MASTER_IP        MACHINE_TYPE   NODE_VERSION    NUM_NODES  STATUS
e2e-be13a81-3027  us-east1-b  1.14.10-gke.42  104.196.151.217  e2-standard-8  1.14.10-gke.42  2          RUNNING
++ gcloud container clusters get-credentials e2e-be13a81-3027
Fetching cluster endpoint and auth data.
kubeconfig entry generated for e2e-be13a81-3027.
++ kubectl delete namespace kubeflow --wait
Error from server (NotFound): namespaces "kubeflow" not found
++ echo 'No need to delete kubeflow namespace. It doesn'\''t exist.'
No need to delete kubeflow namespace. It doesn't exist.
++ kubectl create namespace kubeflow --dry-run -o yaml
++ kubectl apply -f -
W0702 03:08:09.447536    1672 helpers.go:535] --dry-run is deprecated and can be replaced with --dry-run=client.
namespace/kubeflow created
... skipping 534 lines ...
Step #1 - "verify":     title: Database password (Managed storage only)
Step #1 - "verify":     x-google-marketplace:
Step #1 - "verify":       type: MASKED_FIELD
Step #1 - "verify":     description: |-
Step #1 - "verify":       The database password to use when connecting to the Cloud SQL instance.
Step #1 - "verify":       If you leave this field empty, the deployment will try to connect to the instance without providing a password.
Step #1 - "verify":       This will fail if a password is required for the username you provided.
Step #1 - "verify":   managedstorage.databaseNamePrefix:
Step #1 - "verify":     type: string
Step #1 - "verify":     title: Database name prefix (Managed storage only)
Step #1 - "verify":     description: |-
Step #1 - "verify":       The prefix of the database name. Kubeflow Pipelines will create two databases,
Step #1 - "verify":       [prefix]_pipeline and [prefix]_metadata.
... skipping 591 lines ...
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify": Elapsed 0s
Elapsed 5s
Elapsed 9s
Elapsed 13s
Elapsed 18s
Elapsed 22s
Elapsed 27s
Elapsed 31s
Elapsed 36s
Elapsed 40s
Elapsed 44s
Elapsed 49s
Elapsed 53s
Elapsed 58s
Elapsed 62s
Elapsed 67s
Elapsed 71s
Elapsed 75s
Elapsed 80s
Elapsed 84s
Elapsed 89s
Elapsed 93s
Elapsed 98s
Elapsed 102s
Elapsed 106s
Elapsed 111s
Elapsed 115s
Elapsed 120s
Elapsed 124s
Elapsed 129s
Elapsed 133s
Elapsed 137s
Elapsed 142s
Elapsed 146s
Elapsed 151s
Elapsed 155s
Elapsed 160s
Elapsed 164s
Elapsed 168s
Elapsed 173s
Elapsed 177s
Elapsed 182s
Elapsed 186s
Elapsed 190s
Elapsed 195s
Elapsed 199s
Elapsed 204s
Elapsed 208s
Elapsed 213s
Elapsed 217s
Elapsed 221s
Elapsed 226s
Elapsed 230s
Elapsed 235s
Elapsed 239s
Elapsed 243s
Elapsed 248s
Elapsed 252s
Elapsed 257s
Elapsed 261s
Elapsed 266s
Elapsed 270s
Elapsed 274s
Elapsed 279s
Elapsed 283s
Elapsed 288s
Elapsed 292s
Elapsed 297s
Elapsed 301s
Elapsed 305s
Elapsed 310s
Elapsed 314s
Elapsed 319s
Elapsed 323s
Elapsed 327s
Elapsed 332s
Elapsed 336s
Elapsed 341s
Elapsed 345s
Elapsed 350s
Elapsed 354s
Elapsed 358s
Elapsed 363s
Elapsed 367s
Elapsed 372s
Elapsed 376s
Elapsed 381s
Elapsed 385s
Elapsed 390s
Elapsed 394s
Elapsed 398s
Elapsed 403s
Elapsed 407s
Elapsed 412s
Elapsed 416s
Elapsed 421s
Elapsed 425s
Elapsed 429s
Elapsed 434s
Elapsed 438s
Elapsed 443s
Elapsed 447s
Elapsed 451s
Elapsed 456s
Elapsed 460s
Elapsed 465s
Elapsed 469s
Elapsed 474s
Elapsed 478s
Elapsed 482s
Elapsed 487s
Elapsed 491s
Elapsed 496s
Elapsed 500s
Elapsed 505s
Elapsed 509s
Elapsed 513s
Elapsed 518s
Elapsed 522s
Elapsed 527s
Elapsed 531s
Elapsed 536s
Elapsed 540s
Elapsed 544s
Elapsed 549s
Elapsed 553s
Elapsed 558s
Elapsed 562s
Elapsed 567s
Elapsed 571s
Elapsed 575s
Elapsed 580s
Elapsed 584s
Elapsed 589s
Elapsed 593s
Elapsed 598s
Elapsed 602s
Elapsed 607s
Elapsed 611s
Elapsed 616s
Elapsed 620s
Elapsed 624s
Elapsed 629s
Elapsed 633s
Elapsed 638s
Elapsed 642s
Elapsed 646s
Elapsed 651s
Elapsed 655s
Elapsed 660s
Elapsed 664s
Elapsed 669s
Elapsed 673s
Elapsed 677s
Elapsed 682s
Elapsed 686s
Elapsed 691s
Elapsed 695s
Elapsed 700s
Elapsed 704s
Elapsed 708s
Elapsed 713s
Elapsed 717s
Elapsed 722s
Elapsed 726s
Elapsed 731s
Elapsed 735s
Elapsed 739s
Elapsed 744s
Elapsed 748s
Elapsed 753s
Elapsed 757s
Elapsed 761s
Elapsed 766s
Elapsed 770s
Elapsed 775s
Elapsed 779s
Elapsed 784s
Elapsed 788s
Elapsed 792s
Elapsed 797s
Elapsed 801s
Elapsed 806s
Elapsed 810s
Elapsed 815s
Elapsed 819s
Elapsed 823s
Elapsed 828s
Elapsed 832s
Elapsed 837s
Elapsed 841s
Elapsed 846s
Elapsed 850s
Elapsed 854s
Elapsed 859s
Elapsed 863s
Elapsed 868s
Elapsed 872s
Elapsed 877s
Elapsed 881s
Elapsed 885s
Elapsed 890s
Elapsed 894s
Elapsed 899s
Elapsed 903s
Elapsed 908s
Elapsed 912s
Elapsed 916s
Elapsed 921s
Elapsed 925s
Elapsed 930s
Elapsed 934s
Elapsed 939s
Elapsed 943s
Elapsed 947s
Elapsed 952s
Elapsed 956s
Elapsed 961s
Elapsed 965s
Elapsed 970s
Elapsed 974s
Elapsed 978s
Elapsed 983s
Elapsed 987s
Elapsed 992s
Elapsed 996s
Elapsed 1001s
Elapsed 1005s
Elapsed 1009s
Elapsed 1014s
Elapsed 1018s
Elapsed 1023s
Elapsed 1027s
Elapsed 1032s
Elapsed 1036s
Elapsed 1040s
Elapsed 1045s
Elapsed 1049s
Elapsed 1054s
Elapsed 1058s
Elapsed 1063s
Elapsed 1067s
Elapsed 1072s
Elapsed 1076s
Elapsed 1080s
Elapsed 1085s
Elapsed 1089s
Elapsed 1094s
Elapsed 1098s
Elapsed 1103s
Elapsed 1107s
Elapsed 1111s
Elapsed 1116s
Elapsed 1120s
Elapsed 1125s
Elapsed 1129s
Elapsed 1133s
Elapsed 1138s
Elapsed 1142s
Elapsed 1147s
Elapsed 1151s
Elapsed 1156s
Elapsed 1160s
Elapsed 1164s
Elapsed 1169s
Elapsed 1173s
Elapsed 1178s
Elapsed 1182s
Elapsed 1187s
Elapsed 1191s
Elapsed 1195s
Elapsed 1200s
Elapsed 1205s
Elapsed 1209s
Elapsed 1213s
Elapsed 1218s
Elapsed 1222s
Elapsed 1227s
Elapsed 1231s
Elapsed 1236s
Elapsed 1240s
Elapsed 1244s
Elapsed 1249s
Elapsed 1253s
Elapsed 1258s
Elapsed 1262s
Elapsed 1266s
Elapsed 1271s
Elapsed 1275s
Elapsed 1280s
Elapsed 1284s
Elapsed 1289s
Elapsed 1293s
Elapsed 1297s
Elapsed 1302s
Elapsed 1306s
Elapsed 1311s
Elapsed 1315s
Elapsed 1320s
Elapsed 1324s
Elapsed 1328s
Elapsed 1333s
Elapsed 1337s
Elapsed 1342s
Elapsed 1346s
Elapsed 1351s
Elapsed 1355s
Elapsed 1359s
Elapsed 1364s
Elapsed 1368s
Elapsed 1373s
Elapsed 1377s
Elapsed 1382s
Elapsed 1386s
Elapsed 1390s
Elapsed 1395s
Elapsed 1399s
Elapsed 1404s
Elapsed 1408s
Elapsed 1413s
Elapsed 1417s
Elapsed 1421s
Elapsed 1426s
Elapsed 1430s
Elapsed 1435s
Elapsed 1439s
Elapsed 1444s
Elapsed 1448s
Elapsed 1452s
Elapsed 1457s
Elapsed 1461s
Elapsed 1466s
Elapsed 1470s
Elapsed 1474s
Elapsed 1479s
Elapsed 1483s
Elapsed 1488s
Elapsed 1492s
Elapsed 1497s
Elapsed 1501s
Elapsed 1505s
Elapsed 1510s
Elapsed 1514s
Elapsed 1519s
Elapsed 1523s
Elapsed 1528s
Elapsed 1532s
Elapsed 1536s
Elapsed 1541s
Elapsed 1545s
Elapsed 1550s
Elapsed 1554s
Elapsed 1559s
Elapsed 1563s
Elapsed 1567s
Elapsed 1572s
Elapsed 1576s
Elapsed 1581s
Elapsed 1585s
Elapsed 1590s
Elapsed 1594s
Elapsed 1598s
Elapsed 1603s
Elapsed 1607s
Elapsed 1612s
Elapsed 1616s
Elapsed 1621s
Elapsed 1625s
Elapsed 1629s
Elapsed 1634s
Elapsed 1638s
Elapsed 1643s
Elapsed 1647s
Elapsed 1651s
Elapsed 1656s
Elapsed 1660s
Elapsed 1665s
Elapsed 1669s
Elapsed 1674s
Elapsed 1678s
Elapsed 1682s
Elapsed 1687s
Elapsed 1691s
Elapsed 1696s
Elapsed 1700s
Elapsed 1705s
Elapsed 1709s
Elapsed 1714s
Elapsed 1718s
Elapsed 1722s
Elapsed 1727s
Elapsed 1731s
Elapsed 1736s
Elapsed 1740s
Elapsed 1745s
Elapsed 1749s
Elapsed 1753s
Elapsed 1758s
Elapsed 1762s
Elapsed 1767s
Elapsed 1771s
Elapsed 1775s
Elapsed 1780s
Elapsed 1784s
Elapsed 1789s
Elapsed 1793s
Elapsed 1798s
Elapsed 1802s
ERROR Deployer job timeout
Step #1 - "verify": INFO cleaning all resources.
Step #1 - "verify": INFO Collecting events for namespace "apptest-mulgafhl"
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify": LAST SEEN   TYPE      REASON                   OBJECT                                                  MESSAGE
Step #1 - "verify": 30m         Normal    Scheduled                pod/apptest-mulgafhl-deployer-c4g92                     Successfully assigned apptest-mulgafhl/apptest-mulgafhl-deployer-c4g92 to gke-hosted-verify-test-default-pool-53177527-rcfl
Step #1 - "verify": 30m         Normal    Pulling                  pod/apptest-mulgafhl-deployer-c4g92                     Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/deployer:1.0.0-dev.2"
Step #1 - "verify": 29m         Normal    Pulled                   pod/apptest-mulgafhl-deployer-c4g92                     Successfully pulled image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/deployer:1.0.0-dev.2"
Step #1 - "verify": 29m         Normal    Created                  pod/apptest-mulgafhl-deployer-c4g92                     Created container deployer
Step #1 - "verify": 29m         Normal    Started                  pod/apptest-mulgafhl-deployer-c4g92                     Started container deployer
Step #1 - "verify": 30m         Normal    SuccessfulCreate         job/apptest-mulgafhl-deployer                           Created pod: apptest-mulgafhl-deployer-c4g92
Step #1 - "verify": 29m         Normal    Scheduled                pod/cache-deployer-deployment-5bfcb8cb8d-njclh          Successfully assigned apptest-mulgafhl/cache-deployer-deployment-5bfcb8cb8d-njclh to gke-hosted-verify-test-default-pool-53177527-rcfl
Step #1 - "verify": 28m         Normal    Pulling                  pod/cache-deployer-deployment-5bfcb8cb8d-njclh          Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/cachedeployer:0.5.1"
Step #1 - "verify": 28m         Warning   Failed                   pod/cache-deployer-deployment-5bfcb8cb8d-njclh          Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/cachedeployer:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/cachedeployer:0.5.1 not found
Step #1 - "verify": 28m         Warning   Failed                   pod/cache-deployer-deployment-5bfcb8cb8d-njclh          Error: ErrImagePull
Step #1 - "verify": 24m         Normal    BackOff                  pod/cache-deployer-deployment-5bfcb8cb8d-njclh          Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/cachedeployer:0.5.1"
Step #1 - "verify": 4m7s        Warning   Failed                   pod/cache-deployer-deployment-5bfcb8cb8d-njclh          Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/cache-deployer-deployment-5bfcb8cb8d         Created pod: cache-deployer-deployment-5bfcb8cb8d-njclh
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/cache-deployer-deployment                    Scaled up replica set cache-deployer-deployment-5bfcb8cb8d to 1
Step #1 - "verify": 29m         Normal    Scheduled                pod/cache-server-565c867cc6-fgdkj                       Successfully assigned apptest-mulgafhl/cache-server-565c867cc6-fgdkj to gke-hosted-verify-test-default-pool-53177527-dkq2
Step #1 - "verify": 2m47s       Warning   FailedMount              pod/cache-server-565c867cc6-fgdkj                       MountVolume.SetUp failed for volume "webhook-tls-certs" : secret "webhook-server-tls" not found
Step #1 - "verify": 7m1s        Warning   FailedMount              pod/cache-server-565c867cc6-fgdkj                       Unable to mount volumes for pod "cache-server-565c867cc6-fgdkj_apptest-mulgafhl(a7868130-bc11-11ea-b70f-42010a800202)": timeout expired waiting for volumes to attach or mount for pod "apptest-mulgafhl"/"cache-server-565c867cc6-fgdkj". list of unmounted volumes=[webhook-tls-certs]. list of unattached volumes=[webhook-tls-certs kubeflow-pipelines-cache-token-hrhmx]
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/cache-server-565c867cc6                      Created pod: cache-server-565c867cc6-fgdkj
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/cache-server                                 Scaled up replica set cache-server-565c867cc6 to 1
Step #1 - "verify": 29m         Normal    Scheduled                pod/metadata-envoy-5c58d676bb-vmfkh                     Successfully assigned apptest-mulgafhl/metadata-envoy-5c58d676bb-vmfkh to gke-hosted-verify-test-default-pool-53177527-7rzm
Step #1 - "verify": 27m         Normal    Pulling                  pod/metadata-envoy-5c58d676bb-vmfkh                     Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataenvoy:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/metadata-envoy-5c58d676bb-vmfkh                     Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataenvoy:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataenvoy:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/metadata-envoy-5c58d676bb-vmfkh                     Error: ErrImagePull
Step #1 - "verify": 27m         Normal    BackOff                  pod/metadata-envoy-5c58d676bb-vmfkh                     Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataenvoy:0.5.1"
Step #1 - "verify": 4m12s       Warning   Failed                   pod/metadata-envoy-5c58d676bb-vmfkh                     Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/metadata-envoy-5c58d676bb                    Created pod: metadata-envoy-5c58d676bb-vmfkh
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/metadata-envoy                               Scaled up replica set metadata-envoy-5c58d676bb to 1
Step #1 - "verify": 29m         Normal    Scheduled                pod/metadata-grpc-deployment-85f9bd9f77-2dg8c           Successfully assigned apptest-mulgafhl/metadata-grpc-deployment-85f9bd9f77-2dg8c to gke-hosted-verify-test-default-pool-53177527-dkq2
Step #1 - "verify": 27m         Normal    Pulling                  pod/metadata-grpc-deployment-85f9bd9f77-2dg8c           Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataserver:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/metadata-grpc-deployment-85f9bd9f77-2dg8c           Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataserver:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataserver:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/metadata-grpc-deployment-85f9bd9f77-2dg8c           Error: ErrImagePull
Step #1 - "verify": 4m11s       Normal    BackOff                  pod/metadata-grpc-deployment-85f9bd9f77-2dg8c           Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataserver:0.5.1"
Step #1 - "verify": 9m14s       Warning   Failed                   pod/metadata-grpc-deployment-85f9bd9f77-2dg8c           Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/metadata-grpc-deployment-85f9bd9f77          Created pod: metadata-grpc-deployment-85f9bd9f77-2dg8c
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/metadata-grpc-deployment                     Scaled up replica set metadata-grpc-deployment-85f9bd9f77 to 1
Step #1 - "verify": 29m         Normal    Scheduled                pod/metadata-writer-f5449ff5c-gnqq5                     Successfully assigned apptest-mulgafhl/metadata-writer-f5449ff5c-gnqq5 to gke-hosted-verify-test-default-pool-53177527-rcfl
Step #1 - "verify": 27m         Normal    Pulling                  pod/metadata-writer-f5449ff5c-gnqq5                     Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadatawriter:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/metadata-writer-f5449ff5c-gnqq5                     Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadatawriter:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadatawriter:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/metadata-writer-f5449ff5c-gnqq5                     Error: ErrImagePull
Step #1 - "verify": 19m         Normal    BackOff                  pod/metadata-writer-f5449ff5c-gnqq5                     Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadatawriter:0.5.1"
Step #1 - "verify": 4m5s        Warning   Failed                   pod/metadata-writer-f5449ff5c-gnqq5                     Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/metadata-writer-f5449ff5c                    Created pod: metadata-writer-f5449ff5c-gnqq5
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/metadata-writer                              Scaled up replica set metadata-writer-f5449ff5c to 1
Step #1 - "verify": 29m         Warning   FailedScheduling         pod/minio-d9d5d4649-68fr2                               pod has unbound immediate PersistentVolumeClaims (repeated 3 times)
Step #1 - "verify": 29m         Normal    Scheduled                pod/minio-d9d5d4649-68fr2                               Successfully assigned apptest-mulgafhl/minio-d9d5d4649-68fr2 to gke-hosted-verify-test-default-pool-53177527-rcfl
Step #1 - "verify": 29m         Normal    SuccessfulAttachVolume   pod/minio-d9d5d4649-68fr2                               AttachVolume.Attach succeeded for volume "pvc-a819fcd4-bc11-11ea-b70f-42010a800202"
Step #1 - "verify": 27m         Normal    Pulling                  pod/minio-d9d5d4649-68fr2                               Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/minio:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/minio-d9d5d4649-68fr2                               Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/minio:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/minio:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/minio-d9d5d4649-68fr2                               Error: ErrImagePull
Step #1 - "verify": 3m56s       Normal    BackOff                  pod/minio-d9d5d4649-68fr2                               Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/minio:0.5.1"
Step #1 - "verify": 9m3s        Warning   Failed                   pod/minio-d9d5d4649-68fr2                               Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/minio-d9d5d4649                              Created pod: minio-d9d5d4649-68fr2
Step #1 - "verify": 29m         Normal    ProvisioningSucceeded    persistentvolumeclaim/minio-pvc                         Successfully provisioned volume pvc-a819fcd4-bc11-11ea-b70f-42010a800202 using kubernetes.io/gce-pd
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/minio                                        Scaled up replica set minio-d9d5d4649 to 1
Step #1 - "verify": 29m         Normal    Scheduled                pod/ml-pipeline-68759bf859-lltst                        Successfully assigned apptest-mulgafhl/ml-pipeline-68759bf859-lltst to gke-hosted-verify-test-default-pool-53177527-rcfl
Step #1 - "verify": 27m         Normal    Pulling                  pod/ml-pipeline-68759bf859-lltst                        Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/apiserver:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-68759bf859-lltst                        Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/apiserver:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/apiserver:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-68759bf859-lltst                        Error: ErrImagePull
Step #1 - "verify": 9m8s        Normal    BackOff                  pod/ml-pipeline-68759bf859-lltst                        Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/apiserver:0.5.1"
Step #1 - "verify": 4m15s       Warning   Failed                   pod/ml-pipeline-68759bf859-lltst                        Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/ml-pipeline-68759bf859                       Created pod: ml-pipeline-68759bf859-lltst
Step #1 - "verify": 29m         Normal    Scheduled                pod/ml-pipeline-persistenceagent-964f65f5d-qp8kt        Successfully assigned apptest-mulgafhl/ml-pipeline-persistenceagent-964f65f5d-qp8kt to gke-hosted-verify-test-default-pool-53177527-rcfl
Step #1 - "verify": 27m         Normal    Pulling                  pod/ml-pipeline-persistenceagent-964f65f5d-qp8kt        Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/persistenceagent:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-persistenceagent-964f65f5d-qp8kt        Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/persistenceagent:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/persistenceagent:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-persistenceagent-964f65f5d-qp8kt        Error: ErrImagePull
Step #1 - "verify": 9m16s       Normal    BackOff                  pod/ml-pipeline-persistenceagent-964f65f5d-qp8kt        Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/persistenceagent:0.5.1"
Step #1 - "verify": 4m12s       Warning   Failed                   pod/ml-pipeline-persistenceagent-964f65f5d-qp8kt        Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/ml-pipeline-persistenceagent-964f65f5d       Created pod: ml-pipeline-persistenceagent-964f65f5d-qp8kt
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/ml-pipeline-persistenceagent                 Scaled up replica set ml-pipeline-persistenceagent-964f65f5d to 1
Step #1 - "verify": 29m         Normal    Scheduled                pod/ml-pipeline-scheduledworkflow-6f76c97dd6-kfphl      Successfully assigned apptest-mulgafhl/ml-pipeline-scheduledworkflow-6f76c97dd6-kfphl to gke-hosted-verify-test-default-pool-53177527-dkq2
Step #1 - "verify": 27m         Normal    Pulling                  pod/ml-pipeline-scheduledworkflow-6f76c97dd6-kfphl      Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/scheduledworkflow:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-scheduledworkflow-6f76c97dd6-kfphl      Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/scheduledworkflow:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/scheduledworkflow:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-scheduledworkflow-6f76c97dd6-kfphl      Error: ErrImagePull
Step #1 - "verify": 27m         Normal    BackOff                  pod/ml-pipeline-scheduledworkflow-6f76c97dd6-kfphl      Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/scheduledworkflow:0.5.1"
Step #1 - "verify": 4m5s        Warning   Failed                   pod/ml-pipeline-scheduledworkflow-6f76c97dd6-kfphl      Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/ml-pipeline-scheduledworkflow-6f76c97dd6     Created pod: ml-pipeline-scheduledworkflow-6f76c97dd6-kfphl
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/ml-pipeline-scheduledworkflow                Scaled up replica set ml-pipeline-scheduledworkflow-6f76c97dd6 to 1
Step #1 - "verify": 29m         Normal    Scheduled                pod/ml-pipeline-ui-76c485bcc4-zcnf6                     Successfully assigned apptest-mulgafhl/ml-pipeline-ui-76c485bcc4-zcnf6 to gke-hosted-verify-test-default-pool-53177527-dkq2
Step #1 - "verify": 27m         Normal    Pulling                  pod/ml-pipeline-ui-76c485bcc4-zcnf6                     Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/frontend:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-ui-76c485bcc4-zcnf6                     Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/frontend:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/frontend:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-ui-76c485bcc4-zcnf6                     Error: ErrImagePull
Step #1 - "verify": 14m         Normal    BackOff                  pod/ml-pipeline-ui-76c485bcc4-zcnf6                     Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/frontend:0.5.1"
Step #1 - "verify": 4m5s        Warning   Failed                   pod/ml-pipeline-ui-76c485bcc4-zcnf6                     Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/ml-pipeline-ui-76c485bcc4                    Created pod: ml-pipeline-ui-76c485bcc4-zcnf6
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/ml-pipeline-ui                               Scaled up replica set ml-pipeline-ui-76c485bcc4 to 1
Step #1 - "verify": 29m         Normal    Scheduled                pod/ml-pipeline-viewer-crd-5d76ccc7dd-ppt88             Successfully assigned apptest-mulgafhl/ml-pipeline-viewer-crd-5d76ccc7dd-ppt88 to gke-hosted-verify-test-default-pool-53177527-7rzm
Step #1 - "verify": 27m         Normal    Pulling                  pod/ml-pipeline-viewer-crd-5d76ccc7dd-ppt88             Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/viewercrd:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-viewer-crd-5d76ccc7dd-ppt88             Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/viewercrd:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/viewercrd:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-viewer-crd-5d76ccc7dd-ppt88             Error: ErrImagePull
Step #1 - "verify": 24m         Normal    BackOff                  pod/ml-pipeline-viewer-crd-5d76ccc7dd-ppt88             Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/viewercrd:0.5.1"
Step #1 - "verify": 4m4s        Warning   Failed                   pod/ml-pipeline-viewer-crd-5d76ccc7dd-ppt88             Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/ml-pipeline-viewer-crd-5d76ccc7dd            Created pod: ml-pipeline-viewer-crd-5d76ccc7dd-ppt88
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/ml-pipeline-viewer-crd                       Scaled up replica set ml-pipeline-viewer-crd-5d76ccc7dd to 1
Step #1 - "verify": 29m         Normal    Scheduled                pod/ml-pipeline-visualizationserver-796d985b67-mtxx7    Successfully assigned apptest-mulgafhl/ml-pipeline-visualizationserver-796d985b67-mtxx7 to gke-hosted-verify-test-default-pool-53177527-dkq2
Step #1 - "verify": 27m         Normal    Pulling                  pod/ml-pipeline-visualizationserver-796d985b67-mtxx7    Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/visualizationserver:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-visualizationserver-796d985b67-mtxx7    Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/visualizationserver:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/visualizationserver:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/ml-pipeline-visualizationserver-796d985b67-mtxx7    Error: ErrImagePull
Step #1 - "verify": 24m         Normal    BackOff                  pod/ml-pipeline-visualizationserver-796d985b67-mtxx7    Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/visualizationserver:0.5.1"
Step #1 - "verify": 4m11s       Warning   Failed                   pod/ml-pipeline-visualizationserver-796d985b67-mtxx7    Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/ml-pipeline-visualizationserver-796d985b67   Created pod: ml-pipeline-visualizationserver-796d985b67-mtxx7
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/ml-pipeline-visualizationserver              Scaled up replica set ml-pipeline-visualizationserver-796d985b67 to 1
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/ml-pipeline                                  Scaled up replica set ml-pipeline-68759bf859 to 1
Step #1 - "verify": 29m         Warning   FailedScheduling         pod/mysql-8448f4f988-xqh69                              pod has unbound immediate PersistentVolumeClaims (repeated 3 times)
Step #1 - "verify": 29m         Normal    Scheduled                pod/mysql-8448f4f988-xqh69                              Successfully assigned apptest-mulgafhl/mysql-8448f4f988-xqh69 to gke-hosted-verify-test-default-pool-53177527-dkq2
Step #1 - "verify": 29m         Normal    SuccessfulAttachVolume   pod/mysql-8448f4f988-xqh69                              AttachVolume.Attach succeeded for volume "pvc-a825ed2e-bc11-11ea-b70f-42010a800202"
Step #1 - "verify": 27m         Normal    Pulling                  pod/mysql-8448f4f988-xqh69                              Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/mysql:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/mysql-8448f4f988-xqh69                              Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/mysql:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/mysql:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/mysql-8448f4f988-xqh69                              Error: ErrImagePull
Step #1 - "verify": 13m         Normal    BackOff                  pod/mysql-8448f4f988-xqh69                              Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/mysql:0.5.1"
Step #1 - "verify": 4m2s        Warning   Failed                   pod/mysql-8448f4f988-xqh69                              Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/mysql-8448f4f988                             Created pod: mysql-8448f4f988-xqh69
Step #1 - "verify": 29m         Normal    ProvisioningSucceeded    persistentvolumeclaim/mysql-pv-claim                    Successfully provisioned volume pvc-a825ed2e-bc11-11ea-b70f-42010a800202 using kubernetes.io/gce-pd
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/mysql                                        Scaled up replica set mysql-8448f4f988 to 1
Step #1 - "verify": 29m         Normal    Scheduled                pod/proxy-agent-64dfd9fc8f-r8zpr                        Successfully assigned apptest-mulgafhl/proxy-agent-64dfd9fc8f-r8zpr to gke-hosted-verify-test-default-pool-53177527-rcfl
Step #1 - "verify": 27m         Normal    Pulling                  pod/proxy-agent-64dfd9fc8f-r8zpr                        Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/proxyagent:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/proxy-agent-64dfd9fc8f-r8zpr                        Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/proxyagent:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/proxyagent:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/proxy-agent-64dfd9fc8f-r8zpr                        Error: ErrImagePull
Step #1 - "verify": 9m11s       Normal    BackOff                  pod/proxy-agent-64dfd9fc8f-r8zpr                        Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/proxyagent:0.5.1"
Step #1 - "verify": 4m19s       Warning   Failed                   pod/proxy-agent-64dfd9fc8f-r8zpr                        Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/proxy-agent-64dfd9fc8f                       Created pod: proxy-agent-64dfd9fc8f-r8zpr
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/proxy-agent                                  Scaled up replica set proxy-agent-64dfd9fc8f to 1
Step #1 - "verify": 29m         Normal    Scheduled                pod/workflow-controller-9c4556c6c-srvts                 Successfully assigned apptest-mulgafhl/workflow-controller-9c4556c6c-srvts to gke-hosted-verify-test-default-pool-53177527-rcfl
Step #1 - "verify": 27m         Normal    Pulling                  pod/workflow-controller-9c4556c6c-srvts                 Pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/argoworkflowcontroller:0.5.1"
Step #1 - "verify": 27m         Warning   Failed                   pod/workflow-controller-9c4556c6c-srvts                 Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/argoworkflowcontroller:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/argoworkflowcontroller:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed                   pod/workflow-controller-9c4556c6c-srvts                 Error: ErrImagePull
Step #1 - "verify": 14m         Normal    BackOff                  pod/workflow-controller-9c4556c6c-srvts                 Back-off pulling image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/argoworkflowcontroller:0.5.1"
Step #1 - "verify": 4m8s        Warning   Failed                   pod/workflow-controller-9c4556c6c-srvts                 Error: ImagePullBackOff
Step #1 - "verify": 29m         Normal    SuccessfulCreate         replicaset/workflow-controller-9c4556c6c                Created pod: workflow-controller-9c4556c6c-srvts
Step #1 - "verify": 29m         Normal    ScalingReplicaSet        deployment/workflow-controller                          Scaled up replica set workflow-controller-9c4556c6c to 1
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify": INFO Collecting logs for deployer
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify":   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Step #1 - "verify":                                  Dload  Upload   Total   Spent    Left  Speed
Step #1 - "verify": 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100    16  100    16    0     0    247      0 --:--:-- --:--:-- --:--:--   253
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify": Error from server (NotFound): configmaps "gcp-default-config" not found
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify": configmap/gcp-default-config created
Step #1 - "verify": + trap handle_failure EXIT
Step #1 - "verify": + LOG_SMOKE_TEST=SMOKE_TEST
Step #1 - "verify": + test_schema=/data-test/schema.yaml
Step #1 - "verify": + overlay_test_schema.py --test_schema /data-test/schema.yaml --original_schema /data/schema.yaml --output /data/schema.yaml
... skipping 327 lines ...
Step #1 - "verify": Marking deployment of application "apptest-mulgafhl" as "Success".
Step #1 - "verify": + for i in "$@"
Step #1 - "verify": + case $i in
Step #1 - "verify": + status=Success
Step #1 - "verify": + shift
Step #1 - "verify": + [[ -z Success ]]
Step #1 - "verify": + [[ Success =~ ^(Pending|Success|Failed)$ ]]
Step #1 - "verify": + [[ -z apptest-mulgafhl ]]
Step #1 - "verify": + [[ -z apptest-mulgafhl ]]
Step #1 - "verify": + echo 'Marking deployment of application "apptest-mulgafhl" as "Success".'
Step #1 - "verify": + [[ Success == \S\u\c\c\e\s\s ]]
Step #1 - "verify": ++ print_published_version.py --empty_if_not_supported
Step #1 - "verify": + published_version=0.5.1
... skipping 5470 lines ...
Step #1 - "verify": kind: List
Step #1 - "verify": metadata:
Step #1 - "verify":   resourceVersion: ""
Step #1 - "verify":   selfLink: ""
Step #1 - "verify": Using /opt/kubectl/1.14/kubectl (server=1.14)
Step #1 - "verify": namespace "apptest-mulgafhl" deleted
Step #1 - "verify": === ERROR SUMMARY ===
Step #1 - "verify": ERROR Deployer job timeout
Step #1 - "verify": 
Step #1 - "verify": Error events found in namespace "apptest-mulgafhl"
Step #1 - "verify": LAST SEEN   TYPE      REASON             OBJECT                                                 MESSAGE
Step #1 - "verify": 28m         Warning   Failed             pod/cache-deployer-deployment-5bfcb8cb8d-njclh         Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/cachedeployer:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/cachedeployer:0.5.1 not found
Step #1 - "verify": 28m         Warning   Failed             pod/cache-deployer-deployment-5bfcb8cb8d-njclh         Error: ErrImagePull
Step #1 - "verify": 4m7s        Warning   Failed             pod/cache-deployer-deployment-5bfcb8cb8d-njclh         Error: ImagePullBackOff
Step #1 - "verify": 2m47s       Warning   FailedMount        pod/cache-server-565c867cc6-fgdkj                      MountVolume.SetUp failed for volume "webhook-tls-certs" : secret "webhook-server-tls" not found
Step #1 - "verify": 7m1s        Warning   FailedMount        pod/cache-server-565c867cc6-fgdkj                      Unable to mount volumes for pod "cache-server-565c867cc6-fgdkj_apptest-mulgafhl(a7868130-bc11-11ea-b70f-42010a800202)": timeout expired waiting for volumes to attach or mount for pod "apptest-mulgafhl"/"cache-server-565c867cc6-fgdkj". list of unmounted volumes=[webhook-tls-certs]. list of unattached volumes=[webhook-tls-certs kubeflow-pipelines-cache-token-hrhmx]
Step #1 - "verify": 27m         Warning   Failed             pod/metadata-envoy-5c58d676bb-vmfkh                    Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataenvoy:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataenvoy:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/metadata-envoy-5c58d676bb-vmfkh                    Error: ErrImagePull
Step #1 - "verify": 4m12s       Warning   Failed             pod/metadata-envoy-5c58d676bb-vmfkh                    Error: ImagePullBackOff
Step #1 - "verify": 27m         Warning   Failed             pod/metadata-grpc-deployment-85f9bd9f77-2dg8c          Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataserver:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadataserver:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/metadata-grpc-deployment-85f9bd9f77-2dg8c          Error: ErrImagePull
Step #1 - "verify": 9m14s       Warning   Failed             pod/metadata-grpc-deployment-85f9bd9f77-2dg8c          Error: ImagePullBackOff
Step #1 - "verify": 27m         Warning   Failed             pod/metadata-writer-f5449ff5c-gnqq5                    Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadatawriter:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/metadatawriter:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/metadata-writer-f5449ff5c-gnqq5                    Error: ErrImagePull
Step #1 - "verify": 4m5s        Warning   Failed             pod/metadata-writer-f5449ff5c-gnqq5                    Error: ImagePullBackOff
Step #1 - "verify": 29m         Warning   FailedScheduling   pod/minio-d9d5d4649-68fr2                              pod has unbound immediate PersistentVolumeClaims (repeated 3 times)
Step #1 - "verify": 27m         Warning   Failed             pod/minio-d9d5d4649-68fr2                              Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/minio:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/minio:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/minio-d9d5d4649-68fr2                              Error: ErrImagePull
Step #1 - "verify": 9m3s        Warning   Failed             pod/minio-d9d5d4649-68fr2                              Error: ImagePullBackOff
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-68759bf859-lltst                       Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/apiserver:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/apiserver:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-68759bf859-lltst                       Error: ErrImagePull
Step #1 - "verify": 4m15s       Warning   Failed             pod/ml-pipeline-68759bf859-lltst                       Error: ImagePullBackOff
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-persistenceagent-964f65f5d-qp8kt       Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/persistenceagent:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/persistenceagent:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-persistenceagent-964f65f5d-qp8kt       Error: ErrImagePull
Step #1 - "verify": 4m12s       Warning   Failed             pod/ml-pipeline-persistenceagent-964f65f5d-qp8kt       Error: ImagePullBackOff
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-scheduledworkflow-6f76c97dd6-kfphl     Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/scheduledworkflow:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/scheduledworkflow:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-scheduledworkflow-6f76c97dd6-kfphl     Error: ErrImagePull
Step #1 - "verify": 4m5s        Warning   Failed             pod/ml-pipeline-scheduledworkflow-6f76c97dd6-kfphl     Error: ImagePullBackOff
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-ui-76c485bcc4-zcnf6                    Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/frontend:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/frontend:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-ui-76c485bcc4-zcnf6                    Error: ErrImagePull
Step #1 - "verify": 4m5s        Warning   Failed             pod/ml-pipeline-ui-76c485bcc4-zcnf6                    Error: ImagePullBackOff
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-viewer-crd-5d76ccc7dd-ppt88            Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/viewercrd:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/viewercrd:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-viewer-crd-5d76ccc7dd-ppt88            Error: ErrImagePull
Step #1 - "verify": 4m4s        Warning   Failed             pod/ml-pipeline-viewer-crd-5d76ccc7dd-ppt88            Error: ImagePullBackOff
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-visualizationserver-796d985b67-mtxx7   Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/visualizationserver:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.ioERROR: (gcloud.builds.submit) build 13deff0a-4b8b-44bb-8824-02905bd66697 completed with status "FAILURE"
/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/visualizationserver:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/ml-pipeline-visualizationserver-796d985b67-mtxx7   Error: ErrImagePull
Step #1 - "verify": 4m11s       Warning   Failed             pod/ml-pipeline-visualizationserver-796d985b67-mtxx7   Error: ImagePullBackOff
Step #1 - "verify": 29m         Warning   FailedScheduling   pod/mysql-8448f4f988-xqh69                             pod has unbound immediate PersistentVolumeClaims (repeated 3 times)
Step #1 - "verify": 27m         Warning   Failed             pod/mysql-8448f4f988-xqh69                             Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/mysql:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/mysql:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/mysql-8448f4f988-xqh69                             Error: ErrImagePull
Step #1 - "verify": 4m2s        Warning   Failed             pod/mysql-8448f4f988-xqh69                             Error: ImagePullBackOff
Step #1 - "verify": 27m         Warning   Failed             pod/proxy-agent-64dfd9fc8f-r8zpr                       Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/proxyagent:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/proxyagent:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/proxy-agent-64dfd9fc8f-r8zpr                       Error: ErrImagePull
Step #1 - "verify": 4m19s       Warning   Failed             pod/proxy-agent-64dfd9fc8f-r8zpr                       Error: ImagePullBackOff
Step #1 - "verify": 27m         Warning   Failed             pod/workflow-controller-9c4556c6c-srvts                Failed to pull image "gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/argoworkflowcontroller:0.5.1": rpc error: code = Unknown desc = Error response from daemon: manifest for gcr.io/ml-pipeline-test/hosted/be13a819f66b34995b97757648b89fe0e732fa43/argoworkflowcontroller:0.5.1 not found
Step #1 - "verify": 27m         Warning   Failed             pod/workflow-controller-9c4556c6c-srvts                Error: ErrImagePull
Step #1 - "verify": 4m8s        Warning   Failed             pod/workflow-controller-9c4556c6c-srvts                Error: ImagePullBackOff
Step #1 - "verify": =====================
Step #1 - "verify": 
Step #1 - "verify": === VERIFICATION STATUS ===
Step #1 - "verify": FAILED
Step #1 - "verify": ===========================
Finished Step #1 - "verify"
ERROR
ERROR: build step 1 "gcr.io/cloud-marketplace-tools/k8s/dev:latest" failed: step exited with non-zero status: 1
--------------------------------------------------------------------------------

+ clean_up
+ set +e
+ set +x
Status of pods before clean up:
... skipping 19 lines ...