404 describe #3104

jameskerr · 2024-06-18T22:49:47Z

Disable Pagination (i.e., limited to 500 rows of query output in window)
Disable Drill Down (i.e., right-click Pivot to Values)
Disable Enhanced Error Reporting (e.g., red squiggly underlines for syntax errors in the editor)
Disable Correlations views in Details side panel & window

The app in this PR does a feature detection step whenever the lake changes or when it boots up for the first time. It will check if the describe endpoint returns a 404. If so, it will disable the following checkboxes above.

Everything seems to be working as intended in this.

jameskerr · 2024-06-18T22:51:44Z

apps/zui/src/app/lakes/root.tsx

  useLayoutEffect(() => {
    if (status) return
    if (lake) dispatch(updateStatus(lake.id))
+    Active.lake.sync()


Whenever the current lake id changes, or it's connection status changes, sync what info we have with the lake server.

jameskerr · 2024-06-18T22:53:06Z

apps/zui/src/app/lakes/root.tsx

  switch (status) {
    case "disconnected":
-      return <ConnectionError onRetry={() => dispatch(initCurrentTab())} />
+      return <ConnectionError onRetry={() => dispatch(updateStatus(lake.id))} />


initCurrentTab only called this updateStatus function. So I inlined it and removed initCurrentTab.

All of this "lake" status/authentication code needs another look at some point.

jameskerr · 2024-06-18T22:53:41Z

apps/zui/src/js/initializers/init-lake.ts

  const id = lakeId || defaultLake().id
  // This sets up the lake's tabs
+  const lake = Lake.find(id)
+  lake.sync()


As soon as the app boots up, we sync the lake.

jameskerr · 2024-06-18T22:55:02Z

apps/zui/src/js/initializers/initialize.ts

-  api.init(store.dispatch, store.getState)
-  initDOM()
-  initIpcListeners(store)
+  initDomainModels({store})


the Domain Models need to be provisioned with the store variable before we call "initLake". So I re-ordered the steps here.

jameskerr · 2024-06-18T22:55:33Z

apps/zui/src/js/state/Lakes/types.ts

  authData?: AuthData
+  features?: {
+    describe: boolean
+  }


I'm going to add a "features" property the the data we cache about the lake. The only feature right now will be "describe".

jameskerr · 2024-06-18T22:56:15Z

apps/zui/src/models/active.ts

+  static get lake() {
+    const id = this.select(Current.getLakeId)
+    return Lake.find(id)
+  }


This is a shortcut to get the Active lake object. Active.lake

jameskerr · 2024-06-18T22:57:39Z

apps/zui/src/models/lake.ts

+
+type LakeModelAttrs = LakeAttrs & {status: LakeStatus}
+
+export class Lake extends DomainModel<LakeModelAttrs> {


Here we are adding the methods we need to the Lake model.

I've added:

find(id)

client

features

sync()

update

save

isRunning()

jameskerr · 2024-06-18T22:58:09Z

apps/zui/src/models/lake/feature-detector.ts

@@ -0,0 +1,21 @@
+import {Lake} from "../lake"
+
+export class FeatureDetector {


This class is responsible for detecting all the features we want.

jameskerr · 2024-06-18T22:58:52Z

apps/zui/src/views/session-page/handler.ts

    const program = this.select(Current.getQueryText)
    const history = this.select(Current.getHistory)

+    if (!Active.lake.features.describe) {


And finally, if the lake does not have describe, do the minimum required and return early.

jameskerr · 2024-06-18T23:00:02Z

packages/zed-js/src/client/base-client.ts

-      dontRejectError: true,
    });
-    return result.json();
+    if (response.ok || response.status === 400) {


Re-arranged the code to allow for an error response from the server. If the describe endpoint returns code 400, there is a problem with the query, but we don't want to throw an error. We want to report the problem to the user. So we parse the JSON as an expected response.

jameskerr · 2024-06-18T23:01:16Z

packages/zed-js/src/client/base-client.ts

-    duplex?: 'half';
-    dontRejectError?: boolean;
-  }) {
+  protected request(opts: Types.RequestOpts) {


I separated the creation of the request into this request method. The existing send method then calls this.

The describeQuery method only uses request and handles errors in a different way than the other methods.

philrz · 2024-06-20T15:54:39Z

I'm currently testing this branch at commit 62a6084.

The attached video shows a successful walk-through of the functionality that these changes disable when accessing an older Zed lake service. As prep, I have the ZNG format Zed sample data loaded to both the lake running behind Zui (Zed commit 13d15d3) and a separate "remote" Zed v1.14.0 lake I've started on port 8888. As a baseline I first demo each of the four functional areas described by the checkboxes in the opening PR text while accessing the data in the local lake, then I access the v.14.0 lake and confirm that each of these is disabled.

Demo.mp4

jameskerr added 4 commits June 13, 2024 15:07

Remove console.log

1eac49a

Merge remote-tracking branch 'origin/main' into 404-describe

87030b6

Removed not needed initCurrentTab

06a42a3

Detect Describe

71daff4

jameskerr commented Jun 18, 2024

View reviewed changes

jameskerr added 4 commits June 18, 2024 16:06

Only concern yourself with the lake attrs for now

9bca4c5

Only check if you can paginate if the lake has a describe endpoint

3fca224

isGroupBy only works if you have the describe endpoint

1fa56b6

Move some code into new file to avoid circular deps

e9588fc

jameskerr requested review from mattnibs and philrz and removed request for philrz June 18, 2024 23:35

jameskerr added 4 commits June 19, 2024 14:03

Don't sync the lake in the initializers

35de799

Fix logout position

2620dff

only save the lake's attrs no the model

328f0a4

Sync lake properly when current lake changes

31b5d55

This comment was marked as resolved.

Sign in to view

jameskerr added 3 commits June 19, 2024 16:30

Distinguish between updating and adding lakes

725f21f

Don't run uid correlations if there is no poolName preset in the session

d16d2ca

Update the plugins when there is no describe endpoint

62a6084

philrz approved these changes Jun 20, 2024

View reviewed changes

jameskerr merged commit 10d8974 into main Jun 20, 2024

jameskerr deleted the 404-describe branch June 20, 2024 17:02

philrz mentioned this pull request Jun 20, 2024

"Error: 404 page not found" when querying an older Zed lake service #3092

Closed


		type LakeModelAttrs = LakeAttrs & {status: LakeStatus}

		export class Lake extends DomainModel<LakeModelAttrs> {

		@@ -0,0 +1,21 @@
		import {Lake} from "../lake"

		export class FeatureDetector {

404 describe #3104

404 describe #3104

Uh oh!

Conversation

jameskerr commented Jun 18, 2024 • edited by philrz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jameskerr Jun 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jameskerr Jun 18, 2024 • edited by philrz Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

philrz commented Jun 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jameskerr commented Jun 18, 2024 •

edited by philrz

Loading

jameskerr Jun 18, 2024 •

edited

Loading

jameskerr Jun 18, 2024 •

edited by philrz

Loading