Skip to content

Commit 56d645c

Browse files
adchiaadchia
committed
fix: Fixing Web UI, which fails for the SQL registry (#3028)
fix: Fix Feast UI failure with SQL registry Signed-off-by: Danny Chiao <[email protected]>
1 parent 3738e44 commit 56d645c

File tree

7 files changed

+243
-13
lines changed

7 files changed

+243
-13
lines changed

‎ui/CONTRIBUTING.md

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
<h2>Table of contents</h2>
2+
3+
- [General contributor notes](#general-contributor-notes)
4+
- [`feast ui` command](#feast-ui-command)
5+
- [NPM package project structure](#npm-package-project-structure)
6+
- [Tests](#tests)
7+
- [Yarn commands](#yarn-commands)
8+
- [`yarn install`](#yarn-install)
9+
- [`yarn start`](#yarn-start)
10+
- [`yarn test`](#yarn-test)
11+
- [Release process](#release-process)
12+
- [(Advanced) Manually publishing the Feast Package to NPM](#advanced-manually-publishing-the-feast-package-to-npm)
13+
- [Requirements](#requirements)
14+
- [Steps for Publishing](#steps-for-publishing)
15+
16+
# General contributor notes
17+
In this doc, we describe how to contribute both to the Feast Web UI NPM package as well as the embedded Feast UI in the Python SDK (i.e. what's run when you run `feast ui`)
18+
19+
## `feast ui` command
20+
You can see the logic in [../sdk/python/feast/ui](../sdk/python/feast/ui/). This instance is loaded in [../sdk/python/feast/ui_server.py](../sdk/python/feast/ui_server.py).
21+
22+
Under the hood, what happens is that the Feast SDK spins up a server which exposes an endpoint to the registry. It then mounts the UI on the server and points it to fetch data from that registry.
23+
24+
## NPM package project structure
25+
The Web UI is powered by a JSON registry dump from Feast (running `feast registry-dump`). Running `yarn start` launches a UI
26+
powered by test data.
27+
- `public/` contains assets as well as demo data loaded by the Web UI.
28+
- There is a `projects-list.json` which represents all Feast projects the UI shows.
29+
- There is also a `registry.json` which is the registry dump for the feature repo.
30+
- `feature_repo/` contains a sample Feast repo which generates the `registry.json`
31+
- `src/` contains the Web UI source code.
32+
- `src/contexts` has React context objects around project level metadata or registry path metadata to inject into pages. The contexts are static contexts provided by [FeastUISansProviders.tsx](src/FeastUISansProviders.tsx)
33+
- `src/parsers` parses the `registry.json` into in memory representations of Feast objects (feature views, data sources, entities, feature services).
34+
- This has ~1:1 mappings to the protobuf objects in [feast/protos/feast/core](https://github.com/feast-dev/feast/tree/master/protos/feast/core).
35+
- There are also "relationships" which create an in-memory lineage graph which can be used to construct links in pages.
36+
- This generates state which pages will load via React queries (to the registry path).
37+
- `src/pages` has all individual web pages and their layouts. For any given Feast object (e.g. entity), there exist:
38+
- an **Index page** (which is the first page you hit when you click on that object). This loads using a React query the in memory representation of all objects (parsed from `src/parsers`) and embeds:
39+
- a **Listing page** (i.e. listing all the objects in the registry in a table). This creates links to the instance pages
40+
- an **Instance page** (which shows details for an individual entity, feature view, etc). This embeds:
41+
- a default Overview tab, which shows all the Feast metadata (e.g. for a given entity)
42+
- custom tabs from `src/custom-tabs`.
43+
- Other subdirectories:
44+
- `src/components` has common React components that are re-used across pages
45+
- `src/custom-tabs` houses custom tabs and a custom tab React context which exist on the core pages. There is a `TabsRegistryContext` which is also supplied by the [FeastUISansProviders.tsx](src/FeastUISansProviders.tsx), and if there are custom tabs, the Feast UI will embed them as a new tab in the corresponding page (e.g. feature view page).
46+
- `src/graphics` houses icons that are used throughout the UI
47+
- `src/hooks` has React hooks. The most complex hooks here define the bulk of the search / filter functionality.
48+
49+
## Tests
50+
There are very few tests for this UI. There is a smoke test that ensures pages can load in [FeastUISansProviders.test.tsx](src/FeastUISansProviders.test.tsx)
51+
52+
53+
## Yarn commands
54+
55+
If you would like to simply try things out and see how the UI works, you can simply run the code in this repo.
56+
57+
> **Note**: there is an `.npmrc` which is setup for automatic releases. You'll need to comment out the line in there and continue
58+
59+
First:
60+
61+
### `yarn install`
62+
63+
That will install the all the dependencies that the UI needs, as well as development dependencies. Then in the project directory, you can run:
64+
65+
### `yarn start`
66+
67+
Runs the app in the development mode.\
68+
Open [http://localhost:3000](http://localhost:3000) to view it in the browser.
69+
70+
The page will reload if you make edits.\
71+
You will also see any lint errors in the console.
72+
73+
### `yarn test`
74+
75+
Launches the test runner in the interactive watch mode.\
76+
See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.
77+
78+
# Release process
79+
There are a couple of components in Feast that are tied to the Web UI. These are all automatically handled during the release GitHub action:
80+
1. the npm package
81+
- The release process for Feast automatically bumps the package version (see [bump_file_versions.py](../infra/scripts/release/bump_file_versions.py)) and releases the new NPM package (see [publish.yml](../.github/workflows/publish.yml) in the `publish-web-ui-npm` job)
82+
2. the Feast Python SDK, which bundles in a compiled version of the Feast Web UI which is run on a `feast ui` CLI command.
83+
- The bundled Web UI in the Python SDK always compiles in the latest npm version
84+
85+
## (Advanced) Manually publishing the Feast Package to NPM
86+
87+
This generally should not be necessary, since new package versions are released with the overall Feast release workflow (see [publish.yml](../.github/workflows/publish.yml) in the `publish-web-ui-npm` job)
88+
89+
The Feast UI is published as a module to NPM and can be found here: https://www.npmjs.com/package/@feast-dev/feast-ui
90+
91+
### Requirements
92+
93+
To publish a new version of the module, you will need:
94+
- to be part of the @feast-dev team in NPM. Ask `#feast-development` on http://slack.feast.dev to add you if necessary.
95+
- to [login to your NPM account on the command line](https://docs.npmjs.com/cli/v8/commands/npm-adduser).
96+
97+
### Steps for Publishing
98+
99+
1. Make sure tests are passing. Run tests with `yarn test` in the ui directory.
100+
2. Bump the version number in `package.json` as appropriate.
101+
3. Package the modules for distributions. Run the library build script with `yarn build:lib`. We use [Rollup](https://rollupjs.org/) for building the module, and the configs are in the `rollup.config.js` file.
102+
4. Publish the package to NPM. Run `npm publish`
103+
5. [Check NPM to see that the package was properly published](https://www.npmjs.com/package/@feast-dev/feast-ui).

‎ui/src/custom-tabs/data-tab/DataQuery.tsx

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
import { useQuery } from "react-query";
2+
3+
const DataQuery = (featureView: string) => {
4+
const queryKey = `data-tab-namespace:${featureView}`;
5+
6+
return useQuery<any>(
7+
queryKey,
8+
() => {
9+
// Customizing the URL based on your needs
10+
const url = `/demo-custom-tabs/demo.json`;
11+
12+
return fetch(url).then((res) => res.json());
13+
},
14+
{
15+
enabled: !!featureView, // Only start the query when the variable is not undefined
16+
}
17+
);
18+
};
19+
20+
export default DataQuery;

‎ui/src/custom-tabs/data-tab/DataTab.tsx

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
import React from "react";
2+
import { z } from "zod";
3+
import {
4+
EuiCode,
5+
EuiFlexGroup,
6+
EuiHorizontalRule,
7+
EuiLoadingSpinner,
8+
EuiTable,
9+
EuiTitle,
10+
EuiTableHeader,
11+
EuiTableHeaderCell,
12+
EuiPanel,
13+
EuiFlexItem,
14+
EuiTableRow,
15+
EuiTableRowCell,
16+
} from "@elastic/eui";
17+
import DataQuery from "./DataQuery";
18+
19+
const FeatureViewDataRow = z.object({
20+
name: z.string(),
21+
value: z.string(),
22+
});
23+
24+
type FeatureViewDataRowType = z.infer<typeof FeatureViewDataRow>;
25+
26+
const LineHeightProp: React.CSSProperties = {
27+
lineHeight: 1,
28+
};
29+
30+
const EuiFeatureViewDataRow = ({ name, value }: FeatureViewDataRowType) => {
31+
return (
32+
<EuiTableRow>
33+
<EuiTableRowCell>{name}</EuiTableRowCell>
34+
<EuiTableRowCell textOnly={false}>
35+
<EuiCode data-code-language="text">
36+
<pre style={LineHeightProp}>{value}</pre>
37+
</EuiCode>
38+
</EuiTableRowCell>
39+
</EuiTableRow>
40+
);
41+
};
42+
43+
const FeatureViewDataTable = (data: any) => {
44+
var items: FeatureViewDataRowType[] = [];
45+
46+
for (let element in data.data) {
47+
const row: FeatureViewDataRowType = {
48+
name: element,
49+
value: JSON.stringify(data.data[element], null, 2),
50+
};
51+
items.push(row);
52+
console.log(row);
53+
}
54+
55+
return (
56+
<EuiTable>
57+
<EuiTableHeader>
58+
<EuiTableHeaderCell>Data Item Name</EuiTableHeaderCell>
59+
<EuiTableHeaderCell>Data Item Value</EuiTableHeaderCell>
60+
</EuiTableHeader>
61+
{items.map((item) => {
62+
return <EuiFeatureViewDataRow name={item.name} value={item.value} />;
63+
})}
64+
</EuiTable>
65+
);
66+
};
67+
68+
const DataTab = () => {
69+
const fName = "credit_history";
70+
const { isLoading, isError, isSuccess, data } = DataQuery(fName);
71+
const isEmpty = data === undefined;
72+
73+
return (
74+
<React.Fragment>
75+
{isLoading && (
76+
<React.Fragment>
77+
<EuiLoadingSpinner size="m" /> Loading
78+
</React.Fragment>
79+
)}
80+
{isEmpty && <p>No feature view with name: {fName}</p>}
81+
{isError && <p>Error loading feature view: {fName}</p>}
82+
{isSuccess && data && (
83+
<React.Fragment>
84+
<EuiFlexGroup>
85+
<EuiFlexItem>
86+
<EuiPanel hasBorder={true}>
87+
<EuiTitle size="xs">
88+
<h3>Properties</h3>
89+
</EuiTitle>
90+
<EuiHorizontalRule margin="xs" />
91+
<FeatureViewDataTable data={data} />
92+
</EuiPanel>
93+
</EuiFlexItem>
94+
</EuiFlexGroup>
95+
</React.Fragment>
96+
)}
97+
</React.Fragment>
98+
);
99+
};
100+
101+
export default DataTab;

‎ui/src/pages/feature-services/FeatureServiceListingTable.tsx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,10 +53,10 @@ const FeatureServiceListingTable = ({
5353
},
5454
},
5555
{
56-
name: "Created at",
57-
field: "meta.createdTimestamp",
56+
name: "Last updated",
57+
field: "meta.lastUpdatedTimestamp",
5858
render: (date: Date) => {
59-
return date.toLocaleDateString("en-CA");
59+
return date ? date.toLocaleDateString("en-CA") : "n/a";
6060
},
6161
},
6262
];

‎ui/src/pages/feature-services/FeatureServiceOverviewTab.tsx

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -66,14 +66,20 @@ const FeatureServiceOverviewTab = () => {
6666
description="Feature Views"
6767
/>
6868
</EuiFlexItem>
69-
<EuiFlexItem>
70-
<EuiStat
71-
title={`${data.meta.createdTimestamp.toLocaleDateString(
72-
"en-CA"
73-
)}`}
74-
description="Created"
75-
/>
76-
</EuiFlexItem>
69+
{data.meta.lastUpdatedTimestamp ? (
70+
<EuiFlexItem>
71+
<EuiStat
72+
title={`${data.meta.lastUpdatedTimestamp.toLocaleDateString(
73+
"en-CA"
74+
)}`}
75+
description="Last updated"
76+
/>
77+
</EuiFlexItem>
78+
) : (
79+
<EuiText>
80+
No last updated timestamp specified on this feature service.
81+
</EuiText>
82+
)}
7783
</EuiFlexGroup>
7884
<EuiFlexGroup>
7985
<EuiFlexItem grow={2}>

‎ui/src/parsers/feastFeatureServices.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,8 @@ const FeastFeatureServiceSchema = z.object({
1919
description: z.string().optional(),
2020
}),
2121
meta: z.object({
22-
createdTimestamp: z.string().transform((val) => new Date(val)),
22+
createdTimestamp: z.string().transform((val) => new Date(val)).optional(),
23+
lastUpdatedTimestamp: z.string().transform((val) => new Date(val)).optional(),
2324
}),
2425
});
2526

‎ui/src/parsers/feastODFVS.ts

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
11
import { z } from "zod";
22
import { FeastFeatureColumnSchema } from "./feastFeatureViews";
3-
import { FEAST_FEATURE_VALUE_TYPES } from "./types";
43

54
const FeatureViewProjectionSchema = z.object({
65
featureViewProjection: z.object({

0 commit comments

Comments
 (0)