diff --git a/SPEC.md b/SPEC.md index ba0da60..6dc4098 100644 --- a/SPEC.md +++ b/SPEC.md @@ -302,11 +302,12 @@ There are no templates any more: `launch.sh` and `.desktop` are not generated (2 "prefix_arch": "win32", "user_data": ["SAVES"], "dll_overrides": ["dinput8"], + "controller_layout": null, "generator": "plumber 1.0.0" } ``` -There is no Proton version here: it is asked for at install time (2.8). `user_data` serves two purposes: skipping those folders when installing over an existing copy, and `mkdir -p` on a clean install (5.1 step 3). `dll_overrides` is the decision from 7.3.1. The recipe is needed for rebuilding (7.7) and for debugging. +There is no Proton version here: it is asked for at install time (2.8). `user_data` serves two purposes: skipping those folders when installing over an existing copy, and `mkdir -p` on a clean install (5.1 step 3). `dll_overrides` is the decision from 7.3.1. `controller_layout` is the Workshop ID of a controller layout, or null (7.8). The recipe is needed for rebuilding (7.7) and for debugging. --- @@ -490,7 +491,7 @@ The consequences: `protontricks` finds the game the normal way (it has both the - `--dry-run` prints every path and command and does nothing `[MVP]`. **Done and verified** - `--uninstall` is **done**; it turned out to be needed earlier than `[v2]`: without it there was nothing to prepare a machine for testing the installer with -- `--purge` removes the game folder and the `compatdata/` prefix together with the shortcut, and only from `//game` that is neither a link nor the home folder. The uninstaller deletes the folder it sits in, and a copy carried elsewhere along with its `.plumber` would take that folder with it; anything else stops the uninstall before Steam is touched (exit code 5) +- `--purge` removes the game folder, the `compatdata/` prefix and the game's `shadercache/` together with the shortcut, and only from `//game` that is neither a link nor the home folder. The uninstaller deletes the folder it sits in, and a copy carried elsewhere along with its `.plumber` would take that folder with it; anything else stops the uninstall before Steam is touched (exit code 5). The `` folder goes too, but only when nothing else is in it. **Found on a Deck:** after every purge the shader cache and an empty `` were left behind and had to be deleted by hand **Uninstall looks the entry up by `Exe`, not by appid.** That came out on the very first run: a shortcut added through the Steam UI has a random appid (2.3), and the one computed by our formula does not match it. By `Exe` the entry is found no matter who created it; the real appid is then taken from it, and the `CompatToolMapping`, the artwork and the prefix are removed by that. An entry with no appid at all, rewritten by some other tool, keeps the computed one: its 0 would match every entry without an appid, and `CompatToolMapping "0"` is the Steam Play default for all other games. @@ -809,6 +810,24 @@ Drop a finished `.run` back in → read `recipe.json` from the first 72 KB → r `[?]` A separate button in the header, or the same drop zone as for the archive? +### 7.8 Controller layout + +Feedback from a player: of the games added, only Myst worked with the controller out of the box. Syberia wanted a real mouse, and in Deus Ex the controller could turn, switch weapons and fire, but not walk. A game made for keyboard and mouse needs a layout that maps the controller to them, and Steam keeps one per game. + +**Where Steam keeps the choice.** In `steamapps/common/Steam Controller Configs//config/configset_controller_.vdf`, a file per controller type (`neptune` for the built-in one, `xboxone`, `ps5`, `triton` and so on), whose block `controller_config` maps a game to its layout. A non-Steam game is keyed by its name in lower case, without `/\?%*:|"<>.`, and a layout from the Workshop is `"workshop" ""`. That is how Steam ROM Manager sets layouts for the shortcuts it makes (its `src/lib/controller-manager.ts`). `shortcuts.vdf` has no place for it (8). + +**Community layouts of non-Steam games go by name too.** Steam shows a shortcut the layouts others published for a shortcut of exactly the same name, and SteamInputDB lists them next to those of the Steam version: **verified**, a search for "Deus Ex" found layouts for "Deus Ex: Game of the Year Edition" and for shortcuts named `deus ex` and `deus ex™ goty edition`. The keys of a game are the same in both, so either kind fits. + +**The site cannot ask anyone.** **Verified:** Steam's Web API answers without CORS headers, its search of the Workshop wants a key, and the API of SteamInputDB wants a key and lets in only its own site. The section starts with a box, off by default: a game that supports a controller needs nothing, and the field for the link shows only once the box is ticked. Then the person finds a layout on SteamInputDB, through a link next to the field that searches for the title, and pastes its link. The site takes the ID out of the link and puts it into `recipe.json` as `controller_layout`. A layout's page on SteamInputDB is `steaminputdb.com/config/`, and a bare ID is taken too. The Workshop's own page of a layout is no source of links: **verified**, Steam Community answers "The item is either marked as hidden or you do not have permission to view it" to a browser. Anything else is marked in the field. With the box ticked the build waits for a layout; unticked, whatever the field holds stays out of the recipe. Another archive unticks the box and empties the field: the layout was picked for the last game. + +**The installer asks Steam one thing: which controller the layout is made for.** `ISteamRemoteStorage/GetPublishedFileDetails` needs no key, and Python is not a browser. The type comes as a tag: **verified**, `controller_neptune` on two layouts for Deus Ex. It is asked while Steam is still up; with Steam down, the installer backs the controller's file up and writes `"workshop" ""` for the game into it, next to the other edits. Without a connection the game installs all the same, and the last window says the layout was not set: `install.py` writes such things as `NOTE:` lines, and `setup.sh` adds them to the window. + +**After the restart** the installer checks that the entry is still there and says so in the log, and in the window if it is gone, but does not retry over it: a layout is no reason to restart Steam once more. The uninstaller takes the entry out of every controller's file where it still points at the layout the install set, and backs each such file up first, as the install does. + +**Steam fetches the layout by itself and applies it.** **Verified** on a Deck with SteamOS 3.8.28, Deus Ex Revision and the layout `3037080160`: the entry was written while Steam was down, and 16 s after Steam came back the layout was on disk with nothing more from the installer, in `steamapps/workshop/content/241100//` and as `userdata//ugc/referenced//_controller_config.vdf`, 26 KB of a whole layout. Steam did not rewrite the controller's file, and the game played with the layout on its first launch. So the installer downloads nothing, and the `file_url` the details carry stays unused. + +`[?]` Still open: whether Steam Input has to be turned on for the game with a controller other than the built-in one (`UseSteamControllerConfig` in `localconfig.vdf`, which Steam ROM Manager can set). The check above was made with the built-in one. + --- ## 8. Limitations @@ -825,7 +844,7 @@ Drop a finished `.run` back in → read `recipe.json` from the first 72 KB → r `-e %f` runs the file **directly**, so without the bit `execve` refuses. Worse, the menu item **is shown** anyway: `x-shellscript` is a subtype of `x-executable`, so KDE offers an action that is bound to fail. And the message is misleading: konsole prints `Could not find ''`, although the file is there and simply cannot be executed. The person goes looking for a missing file instead of ticking the box - **A double click** without the bit hands the file to a text editor, which does not open four gigabytes at all. So there is not a single built-in hint: the instructions have to come from the application -- **The controller layout** cannot be baked into `shortcuts.vdf`. The most we can do is put a link to a community layout into the instructions +- **The controller layout** cannot be baked into `shortcuts.vdf`, but it has a file of its own, and the installer writes a layout from the Workshop there (7.8) - **A Steam download in progress will be interrupted** by the install (5.1) - **SteamOS has no `/usr/bin/time`.** `bash` has only the `time` keyword, and POSIX `sh` not even that. Durations are measured with `date +%s`. Verified: a `/usr/bin/time` wrapper around the unpacking silently ran nothing and still returned exit code zero - **There is no `bc` either.** Fractional arithmetic goes through `awk`, integer arithmetic through `$(( ))`. Verified on the spot diff --git a/packages/frontend/app/blocks/ArtworkGrid/ArtworkGrid.ts b/packages/frontend/app/blocks/ArtworkGrid/ArtworkGrid.ts index 05da040..3c44c69 100644 --- a/packages/frontend/app/blocks/ArtworkGrid/ArtworkGrid.ts +++ b/packages/frontend/app/blocks/ArtworkGrid/ArtworkGrid.ts @@ -8,7 +8,7 @@ import { type ArtItem, ART_SLOTS, ART_TYPES, - steamGridDbUrl + getSteamGridDbUrl } from '~/services/artwork' import { Artwork$ } from '~/stores/artwork' import { Game$ } from '~/stores/game' @@ -55,7 +55,7 @@ export const ArtworkGrid = component$(({ title: spec.slot, size: `${spec.width}×${spec.height}`, sourceLabel: spec.hint, - sourceUrl: () => steamGridDbUrl(spec.section, $name() || 'game', spec.dimensions), + sourceUrl: () => getSteamGridDbUrl(spec.section, $name() || 'game', spec.dimensions), accept: ART_TYPES.join(','), preview: () => previewOf($slots()[spec.slot]), clearLabel: 'Remove', diff --git a/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.module.css b/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.module.css new file mode 100644 index 0000000..e3af4a1 --- /dev/null +++ b/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.module.css @@ -0,0 +1,8 @@ +.root { + display: grid; + gap: var(--sizeSpace); +} + +.error { + color: var(--colorDanger); +} diff --git a/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.spec.ts b/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.spec.ts new file mode 100644 index 0000000..e6c54af --- /dev/null +++ b/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.spec.ts @@ -0,0 +1,155 @@ +import { + describe, + it, + expect, + afterEach +} from 'vitest' +import { + render, + screen, + fireEvent +} from '@nanoviews/testing-library' +import { + InjectionContext, + inject, + start, + waitTasks +} from 'nanoviews/store' +import { context$ } from 'nanoviews' +import { Game$ } from '~/stores/game' +import { sampleArchive } from '../shared/game.mock.ts' +import { + type ControllerLayoutProps, + ControllerLayout +} from './ControllerLayout.ts' +import styles from './ControllerLayout.module.css' + +const LINK = 'https://www.steaminputdb.com/config/3036980641' +const NOT_A_LINK = "This is not a link to a layout. A layout's page on SteamInputDB looks like " + + 'https://www.steaminputdb.com/config/3036980641.' +// What the page shows next to the block: the title, the form it is in fills it +const stops: (() => void)[] = [] + +// The block over a game of its own, with the archive read the way a drop reads it +async function setup(props?: ControllerLayoutProps) { + const context = new InjectionContext() + const game = inject(Game$, context) + + stops.push(start(game.$name)) + render(() => context$(context)(ControllerLayout(props))) + game.$file(sampleArchive()) + await waitTasks(game.$archive) + + return game +} + +function mappingBox() { + return screen.getByRole('checkbox', { + name: /^Map the controller to keyboard and mouse/ + }) +} + +function linkBox() { + return screen.getByRole('textbox', { + name: 'Link to a layout' + }) +} + +function paste(text: string) { + fireEvent.input(linkBox(), { + target: { + value: text + } + }) +} + +describe('blocks', () => { + describe('ControllerLayout', () => { + afterEach(() => { + stops.splice(0).forEach(stop => stop()) + }) + + it('should leave mapping off and the link field out at first', async () => { + await setup() + + expect(mappingBox().checked).toBe(false) + expect(screen.queryByRole('textbox')).toBeNull() + }) + + it('should bring the link field in once mapping is ticked, and take it away when unticked', async () => { + const game = await setup() + + fireEvent.click(mappingBox()) + + expect(game.$mapController()).toBe(true) + expect(linkBox()).toBeDefined() + + fireEvent.click(mappingBox()) + + expect(screen.queryByRole('textbox')).toBeNull() + }) + + it('should search SteamInputDB for the game, in a tab of its own', async () => { + await setup() + + fireEvent.click(mappingBox()) + + const search = screen.getByRole('link', { + name: 'on SteamInputDB' + }) + + expect(search.href).toBe('https://www.steaminputdb.com/config/search?searchtext=Need%20for%20Speed') + expect(search.target).toBe('_blank') + expect(search.rel).toBe('noopener noreferrer') + }) + + it('should take the layout out of the link pasted', async () => { + const game = await setup() + + fireEvent.click(mappingBox()) + paste(LINK) + + expect(game.$layoutLink()).toBe(LINK) + expect(game.$layoutId()).toBe('3036980641') + expect(linkBox().getAttribute('aria-invalid')).toBe('false') + }) + + it('should say what a link to a layout looks like when something else is pasted', async () => { + await setup() + + fireEvent.click(mappingBox()) + paste('Deus Ex Steam Deck Perfection') + + expect(screen.getByText(NOT_A_LINK)).toBeDefined() + expect(linkBox().getAttribute('aria-invalid')).toBe('true') + expect(screen.queryByRole('link')).toBeNull() + }) + + it('should go back to the search once the field is emptied', async () => { + await setup() + + fireEvent.click(mappingBox()) + paste('Deus Ex Steam Deck Perfection') + paste('') + + expect(screen.queryByText(NOT_A_LINK)).toBeNull() + expect(screen.getByRole('link')).toBeDefined() + }) + + it('should keep the class passed by the consumer', async () => { + await setup({ + class: 'outer' + }) + + expect(mappingBox().closest('.outer')?.classList.contains(styles.root)).toBe(true) + }) + + it('should hand the other attributes to its root', async () => { + await setup({ + title: 'outer' + }) + + expect(mappingBox().closest(`.${styles.root}`)?.getAttribute('title')).toBe('outer') + }) + }) +}) diff --git a/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.stories.ts b/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.stories.ts new file mode 100644 index 0000000..e23e1cb --- /dev/null +++ b/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.stories.ts @@ -0,0 +1,79 @@ +import type { + Decorator, + Meta, + StoryObj +} from '@nanoviews/storybook' +import { inject } from 'nanoviews/store' +import { div } from 'nanoviews' +import { Game$ } from '~/stores/game' +import { withGame } from '../shared/game.mock.ts' +import { ControllerLayout } from './ControllerLayout.ts' + +/** + * Ticks mapping and puts a pasted link into the field. Without an archive in + * the story: a new archive drops the mapping set up for the last game. + * + * @returns The decorator. + */ +function withLink(link: string): Decorator { + return (story) => { + const { + $mapController, + $layoutLink + } = inject(Game$) + + $mapController(true) + $layoutLink(link) + + return story() + } +} + +const meta: Meta = { + title: 'Blocks/ControllerLayout', + component: ControllerLayout, + decorators: [ + // No wider than the column of the page it sits in + story => ( + div({ + style: { + maxWidth: 'var(--sizeContent)' + } + })( + story() + ) + ), + withGame() + ] +} + +export default meta + +type Story = StoryObj + +/** As it first shows: mapping off, for a game that supports a controller. */ +export const Default: Story = {} + +/** Mapping ticked: the field for a link, and a search on SteamInputDB next to it. */ +export const Mapped: Story = { + decorators: [ + withLink(''), + withGame(null) + ] +} + +/** A layout's page on SteamInputDB pasted in. */ +export const WithLink: Story = { + decorators: [ + withLink('https://www.steaminputdb.com/config/3036980641'), + withGame(null) + ] +} + +/** Something else pasted: the field says what a link to a layout looks like. */ +export const NotALink: Story = { + decorators: [ + withLink('Deus Ex Steam Deck Perfection'), + withGame(null) + ] +} diff --git a/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.ts b/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.ts new file mode 100644 index 0000000..420d3e8 --- /dev/null +++ b/packages/frontend/app/blocks/ControllerLayout/ControllerLayout.ts @@ -0,0 +1,99 @@ +import { inject } from 'nanoviews/store' +import { + type Attributes, + a, + component$, + div, + fragment, + span, + if_ +} from 'nanoviews' +import { getLayoutSearchUrl } from '~/services/controller' +import { Game$ } from '~/stores/game' +import { Checkbox } from '~/uikit/Checkbox' +import { ChoiceField } from '~/uikit/ChoiceField' +import { + Field, + FieldControl +} from '~/uikit/Field' +import { TextInput } from '~/uikit/TextInput' +import mixins from '~/uikit/mixins.module.css' +import styles from './ControllerLayout.module.css' + +export type ControllerLayoutProps = Attributes<'div'> + +/** What a layout's page on SteamInputDB looks like, for whoever pasted something else. */ +const EXAMPLE = 'https://www.steaminputdb.com/config/3036980641' + +/** + * A community layout from the Steam Workshop, for a game made for keyboard and + * mouse. Asked for with a box, off by default since a game that supports a + * controller needs nothing; then the human finds a layout on SteamInputDB and + * pastes its link, and the installer points the game at it (7.8). Neither + * Steam nor SteamInputDB can be asked from a page on another site, so the link + * is all there is to go by. + */ +export const ControllerLayout = component$(({ + class: className, + ...props +}: ControllerLayoutProps) => { + const { + $name, + $mapController, + $layoutLink, + $layoutInvalid + } = inject(Game$) + + return ( + div({ + class: [className, styles.root], + ...props + })( + ChoiceField({ + target: Checkbox({ + checked: $mapController + }), + text: 'Map the controller to keyboard and mouse', + description: 'Leave it off for a game that supports a controller by itself.' + }), + if_($mapController)( + () => ( + Field({ + id: 'game-layout', + label: 'Link to a layout', + hint: if_($layoutInvalid)( + () => span({ + class: styles.error + })( + `This is not a link to a layout. A layout's page on SteamInputDB looks like ${EXAMPLE}.` + ), + () => fragment( + 'Find a layout made for the controller you play with ', + // A tab of its own: leaving the page would drop the archive and + // the artwork already in the form + a({ + class: mixins.focusOutline, + href: () => getLayoutSearchUrl($name() || 'game'), + target: '_blank', + rel: 'noopener noreferrer' + })( + 'on SteamInputDB' + ), + ' and paste its link here. SteamOS needs a connection while the installer sets it.' + ) + ) + })( + FieldControl({ + as: TextInput, + value: $layoutLink, + placeholder: EXAMPLE, + inputMode: 'url', + spellCheck: false, + 'aria-invalid': $layoutInvalid + }) + ) + ) + ) + ) + ) +}) diff --git a/packages/frontend/app/blocks/ControllerLayout/index.ts b/packages/frontend/app/blocks/ControllerLayout/index.ts new file mode 100644 index 0000000..90b0eee --- /dev/null +++ b/packages/frontend/app/blocks/ControllerLayout/index.ts @@ -0,0 +1 @@ +export * from './ControllerLayout.ts' diff --git a/packages/frontend/app/pages/Home/Home.spec.ts b/packages/frontend/app/pages/Home/Home.spec.ts index 2cec5ea..e4973e1 100644 --- a/packages/frontend/app/pages/Home/Home.spec.ts +++ b/packages/frontend/app/pages/Home/Home.spec.ts @@ -157,6 +157,7 @@ describe('pages', () => { 'How it works', 'The game', 'Artwork', + 'Controller', 'Before building' ]) expect(buildButton()).not.toBeNull() @@ -169,6 +170,9 @@ describe('pages', () => { name: 'Title in the library' }))).toBe('The game') expect(headingOver(screen.getByLabelText('portrait, 600×900'))).toBe('Artwork') + expect(headingOver(screen.getByRole('checkbox', { + name: /^Map the controller to keyboard and mouse/ + }))).toBe('Controller') expect(headingOver(screen.getByText(/^5 of 5 artwork slots are empty\./))).toBe('Before building') }) diff --git a/packages/frontend/app/pages/Home/Home.ts b/packages/frontend/app/pages/Home/Home.ts index af2363a..b188d63 100644 --- a/packages/frontend/app/pages/Home/Home.ts +++ b/packages/frontend/app/pages/Home/Home.ts @@ -14,6 +14,7 @@ import { import { ArchivePicker } from '~/blocks/ArchivePicker' import { ArtworkGrid } from '~/blocks/ArtworkGrid' import { BuildPanel } from '~/blocks/BuildPanel' +import { ControllerLayout } from '~/blocks/ControllerLayout' import { GameForm } from '~/blocks/GameForm' import { HowItWorks } from '~/blocks/HowItWorks' import { PreflightList } from '~/blocks/PreflightList' @@ -154,6 +155,13 @@ export const Home = component$(({ })( ArtworkGrid() ), + Section({ + title: 'Controller', + note: 'A game made for keyboard and mouse needs the controller mapped to them, with a ' + + 'community layout from the Steam Workshop.' + })( + ControllerLayout() + ), Section({ title: 'Before building', note: 'Nothing here stops the build.' diff --git a/packages/frontend/app/services/artwork/artwork.spec.ts b/packages/frontend/app/services/artwork/artwork.spec.ts index 22d6901..c5cbc42 100644 --- a/packages/frontend/app/services/artwork/artwork.spec.ts +++ b/packages/frontend/app/services/artwork/artwork.spec.ts @@ -7,7 +7,7 @@ import { ART_SLOTS, ART_TYPES, artFileName, - steamGridDbUrl + getSteamGridDbUrl } from './artwork.ts' const file = (type: string, name = 'art') => new File([new Uint8Array(1)], name, { @@ -34,14 +34,14 @@ describe('services', () => { }) }) - describe('steamGridDbUrl', () => { + describe('getSteamGridDbUrl', () => { it('should open the section filtered by size when it is given', () => { - expect(steamGridDbUrl('grids', 'Need for Speed', '920x430')) + expect(getSteamGridDbUrl('grids', 'Need for Speed', '920x430')) .toBe('https://www.steamgriddb.com/search/grids/920x430/all/all?term=Need%20for%20Speed') }) it('should point at the section with the title as the search term', () => { - expect(steamGridDbUrl('heroes', 'Need for Speed')) + expect(getSteamGridDbUrl('heroes', 'Need for Speed')) .toBe('https://www.steamgriddb.com/search/heroes?term=Need%20for%20Speed') }) }) diff --git a/packages/frontend/app/services/artwork/artwork.ts b/packages/frontend/app/services/artwork/artwork.ts index 23538bb..21f756f 100644 --- a/packages/frontend/app/services/artwork/artwork.ts +++ b/packages/frontend/app/services/artwork/artwork.ts @@ -54,7 +54,7 @@ export const ART_SLOTS: ArtSlotSpec[] = [ * * @returns A search link into the right section of SteamGridDB, filtered by size when a size is given. */ -export function steamGridDbUrl(section: string, term: string, dimensions?: string) { +export function getSteamGridDbUrl(section: string, term: string, dimensions?: string) { const filter = dimensions ? `/${dimensions}/all/all` : '' return `https://www.steamgriddb.com/search/${section}${filter}?term=${encodeURIComponent(term)}` diff --git a/packages/frontend/app/services/bundle/bundle.spec.ts b/packages/frontend/app/services/bundle/bundle.spec.ts index 180ad70..1a8efbe 100644 --- a/packages/frontend/app/services/bundle/bundle.spec.ts +++ b/packages/frontend/app/services/bundle/bundle.spec.ts @@ -21,6 +21,7 @@ const RECIPE = { prefix_arch: 'win32' as const, user_data: ['SAVES'], dll_overrides: ['dinput8'], + controller_layout: null, generator: 'plumber 1.0.0' } diff --git a/packages/frontend/app/services/controller/controller.spec.ts b/packages/frontend/app/services/controller/controller.spec.ts new file mode 100644 index 0000000..8c79222 --- /dev/null +++ b/packages/frontend/app/services/controller/controller.spec.ts @@ -0,0 +1,45 @@ +import { + describe, + it, + expect +} from 'vitest' +import { + getLayoutId, + getLayoutSearchUrl +} from './controller.ts' + +describe('services', () => { + describe('controller', () => { + describe('getLayoutId', () => { + it('should read the ID out of a layout page on SteamInputDB', () => { + expect(getLayoutId('https://www.steaminputdb.com/config/3036980641')).toBe('3036980641') + expect(getLayoutId('https://steaminputdb.com/config/3036980641/')).toBe('3036980641') + expect(getLayoutId(' https://www.steaminputdb.com/config/3036980641?tab=details ')).toBe('3036980641') + }) + + it('should take a bare ID as it is', () => { + expect(getLayoutId(' 3036980641 ')).toBe('3036980641') + }) + + it('should find no ID in an empty field', () => { + expect(getLayoutId(' ')).toBeNull() + }) + + it('should find no ID in a link to anything else', () => { + expect(getLayoutId('https://www.steaminputdb.com/config/search?searchtext=Deus%20Ex')).toBeNull() + expect(getLayoutId('https://www.steaminputdb.com/app/6910')).toBeNull() + expect(getLayoutId('https://example.com/config/3036980641')).toBeNull() + expect(getLayoutId('https://steamcommunity.com/profiles/76561198026632814')).toBeNull() + expect(getLayoutId('ftp://www.steaminputdb.com/config/3036980641')).toBeNull() + expect(getLayoutId('Deus Ex Steam Deck Perfection')).toBeNull() + }) + }) + + describe('getLayoutSearchUrl', () => { + it('should search SteamInputDB for the title', () => { + expect(getLayoutSearchUrl('Deus Ex: GOTY')) + .toBe('https://www.steaminputdb.com/config/search?searchtext=Deus%20Ex%3A%20GOTY') + }) + }) + }) +}) diff --git a/packages/frontend/app/services/controller/controller.ts b/packages/frontend/app/services/controller/controller.ts new file mode 100644 index 0000000..02f109f --- /dev/null +++ b/packages/frontend/app/services/controller/controller.ts @@ -0,0 +1,44 @@ +/** A Workshop item's ID: the number Steam publishes every item under. */ +const FILE_ID = /^\d{1,20}$/ + +/** + * The Workshop ID of a controller layout, out of what the human pasted: the + * layout's page on SteamInputDB, `steaminputdb.com/config/`, or the bare + * ID. Read from the text alone, since neither Steam nor SteamInputDB lets a + * page on another site ask them (7.8). The Workshop itself shows no layout to + * a browser, so there is no link of its own to take one from. + * + * @returns The ID, or null when there is none to be found. + */ +export function getLayoutId(text: string) { + const value = text.trim() + + if (FILE_ID.test(value)) { + return value + } + + let url: URL + + try { + url = new URL(value) + } catch { + return null + } + + if (url.protocol !== 'https:' && url.protocol !== 'http:') { + return null + } + + const id = url.hostname.replace(/^www\./, '') === 'steaminputdb.com' + && /^\/config\/(\d+)\/?$/.exec(url.pathname)?.[1] + + return id && FILE_ID.test(id) ? id : null +} + +/** + * @returns The layouts SteamInputDB has for the game: those of its Steam + * version, and those made for a shortcut under its name. + */ +export function getLayoutSearchUrl(name: string) { + return `https://www.steaminputdb.com/config/search?searchtext=${encodeURIComponent(name)}` +} diff --git a/packages/frontend/app/services/controller/index.ts b/packages/frontend/app/services/controller/index.ts new file mode 100644 index 0000000..8d4ee54 --- /dev/null +++ b/packages/frontend/app/services/controller/index.ts @@ -0,0 +1 @@ +export * from './controller.ts' diff --git a/packages/frontend/app/services/recipe/recipe.spec.ts b/packages/frontend/app/services/recipe/recipe.spec.ts index 21dbfcb..99dc5d5 100644 --- a/packages/frontend/app/services/recipe/recipe.spec.ts +++ b/packages/frontend/app/services/recipe/recipe.spec.ts @@ -27,7 +27,8 @@ const INPUT = { exe: 'Need for Speed Most Wanted/speed.exe', prefixArch: 'win32' as const, userData: ['SAVES'], - dllOverrides: ['dinput8'] + dllOverrides: ['dinput8'], + controllerLayout: '3036980641' } describe('services', () => { @@ -120,6 +121,14 @@ describe('services', () => { expect(recipe.dll_overrides).toEqual([]) }) + it('should carry the Workshop ID of the controller layout, or null for none', () => { + expect(buildRecipe(INPUT).controller_layout).toBe('3036980641') + expect(buildRecipe({ + ...INPUT, + controllerLayout: null + }).controller_layout).toBeNull() + }) + it('should use the key names the installer reads', () => { expect(Object.keys(buildRecipe(INPUT))).toEqual([ 'name', @@ -128,6 +137,7 @@ describe('services', () => { 'prefix_arch', 'user_data', 'dll_overrides', + 'controller_layout', 'generator' ]) }) diff --git a/packages/frontend/app/services/recipe/recipe.ts b/packages/frontend/app/services/recipe/recipe.ts index 94098a9..e7d5fae 100644 --- a/packages/frontend/app/services/recipe/recipe.ts +++ b/packages/frontend/app/services/recipe/recipe.ts @@ -79,6 +79,7 @@ export function buildRecipe(input: RecipeInput): Recipe { prefix_arch: input.prefixArch, user_data: input.userData, dll_overrides: input.dllOverrides, + controller_layout: input.controllerLayout, generator: GENERATOR } } diff --git a/packages/frontend/app/services/recipe/recipe.types.ts b/packages/frontend/app/services/recipe/recipe.types.ts index 3761225..c4e4cd8 100644 --- a/packages/frontend/app/services/recipe/recipe.types.ts +++ b/packages/frontend/app/services/recipe/recipe.types.ts @@ -17,6 +17,12 @@ export interface Recipe { * decision - the human unticked the box - and the installer honours it. */ dll_overrides: string[] + /** + * The Workshop ID of a controller layout for a game made for keyboard and + * mouse, or null. The installer points the game at it, and Steam fetches + * the layout itself (7.8). + */ + controller_layout: string | null generator: string } @@ -26,4 +32,5 @@ export interface RecipeInput { prefixArch: PrefixArch userData: string[] dllOverrides: string[] + controllerLayout: string | null } diff --git a/packages/frontend/app/stores/build.spec.ts b/packages/frontend/app/stores/build.spec.ts index 5aec817..b874446 100644 --- a/packages/frontend/app/stores/build.spec.ts +++ b/packages/frontend/app/stores/build.spec.ts @@ -370,6 +370,48 @@ describe('stores', () => { }) }) + it('should pack the controller layout the human pasted', async () => { + const { game, build } = await setup() + const { written } = pickFile() + + game.$mapController(true) + game.$layoutLink('https://www.steaminputdb.com/config/3036980641') + build.build() + await waitTasks(build.$phase) + + const recipe = filesOf(await unpackMeta(written())).get('./recipe.json') + + expect(JSON.parse(decoder.decode(recipe)).controller_layout).toBe('3036980641') + }) + + it('should not offer to build with mapping on and no layout to map with', async () => { + const { game, build } = await setup() + + game.$mapController(true) + + expect(build.$canBuild()).toBe(false) + + game.$layoutLink('Deus Ex Steam Deck Perfection') + + expect(build.$canBuild()).toBe(false) + }) + + it('should pack no layout while mapping is off, whatever the field holds', async () => { + const { game, build } = await setup() + const { written } = pickFile() + + game.$layoutLink('Deus Ex Steam Deck Perfection') + + expect(build.$canBuild()).toBe(true) + + build.build() + await waitTasks(build.$phase) + + const recipe = filesOf(await unpackMeta(written())).get('./recipe.json') + + expect(JSON.parse(decoder.decode(recipe)).controller_layout).toBeNull() + }) + it('should pack the artwork the build was started with', async () => { const { artwork, build } = await setup() const { written } = pickFile() diff --git a/packages/frontend/app/stores/build.ts b/packages/frontend/app/stores/build.ts index 3ea302c..f533aa6 100644 --- a/packages/frontend/app/stores/build.ts +++ b/packages/frontend/app/stores/build.ts @@ -47,7 +47,9 @@ export function Build$() { $prefixArch, $selected, $overrides, - $userDataList + $userDataList, + $layoutId, + $layoutMissing } = inject(Game$) const { $items, @@ -70,6 +72,7 @@ export function Build$() { && $name().trim() && !$building() && !$problems().length + && !$layoutMissing() )) // A build of an archive that is no longer there writes nothing back let ticket = 0 @@ -96,7 +99,8 @@ export function Build$() { exe: candidate.path, prefixArch: $prefixArch(), userData: $userDataList(), - dllOverrides: $overrides() + dllOverrides: $overrides(), + controllerLayout: $layoutId() }) const fileName = `${slugify(recipe.name)}.run` diff --git a/packages/frontend/app/stores/game.spec.ts b/packages/frontend/app/stores/game.spec.ts index 7528334..670bb17 100644 --- a/packages/frontend/app/stores/game.spec.ts +++ b/packages/frontend/app/stores/game.spec.ts @@ -312,6 +312,66 @@ describe('stores', () => { expect(game.$overrideWrappers()).toBe(false) }) + it('should leave mapping off at first', () => { + const game = createGame() + + expect(game.$mapController()).toBe(false) + expect(game.$layoutMissing()).toBe(false) + }) + + it('should find the layout in the link pasted while mapping is on', () => { + const game = createGame() + + game.$mapController(true) + game.$layoutLink('https://www.steaminputdb.com/config/3036980641') + + expect(game.$layoutId()).toBe('3036980641') + expect(game.$layoutInvalid()).toBe(false) + expect(game.$layoutMissing()).toBe(false) + }) + + it('should give no layout while mapping is off, whatever the field holds', () => { + const game = createGame() + + game.$layoutLink('https://www.steaminputdb.com/config/3036980641') + + expect(game.$layoutId()).toBeNull() + expect(game.$layoutMissing()).toBe(false) + }) + + it('should wait for a layout once mapping is on', () => { + const game = createGame() + + game.$mapController(true) + + expect(game.$layoutMissing()).toBe(true) + }) + + it('should tell text that is not a link to a layout from an empty field', () => { + const game = createGame() + + expect(game.$layoutInvalid()).toBe(false) + + game.$mapController(true) + game.$layoutLink('Deus Ex Steam Deck Perfection') + + expect(game.$layoutId()).toBeNull() + expect(game.$layoutInvalid()).toBe(true) + expect(game.$layoutMissing()).toBe(true) + }) + + it('should drop the mapping set up for the last game when another archive comes', async () => { + const game = await loaded() + + game.$mapController(true) + game.$layoutLink('https://www.steaminputdb.com/config/3036980641') + game.$file(archiveOf('Hot Pursuit 2.zip')) + await waitTasks(game.$archive) + + expect(game.$mapController()).toBe(false) + expect(game.$layoutLink()).toBe('') + }) + it('should leave the form alone while nothing shows it', async () => { const game = createGame() diff --git a/packages/frontend/app/stores/game.ts b/packages/frontend/app/stores/game.ts index 576f4f5..fcff450 100644 --- a/packages/frontend/app/stores/game.ts +++ b/packages/frontend/app/stores/game.ts @@ -6,6 +6,7 @@ import { batch, resolved } from '@nano_kit/store' +import { getLayoutId } from '~/services/controller' import { rankExes } from '~/services/exe' import { type PrefixArch, @@ -68,6 +69,21 @@ export function Game$() { const $userData = signal('SAVES') /** Ticked by default: the detection is right far more often than not (7.3.1). */ const $overrideWrappers = signal(true) + /** + * Whether the controller is to be mapped to keyboard and mouse with a layout + * (7.8). Off by default: a game that supports a controller needs nothing. + */ + const $mapController = signal(false) + /** The link to a layout, as pasted */ + const $layoutLink = signal('') + /** The Workshop ID in the link, whether mapping is on or not */ + const $linkedLayout = computed(() => getLayoutId($layoutLink())) + /** What goes into the recipe: the layout while mapping is on, null otherwise */ + const $layoutId = computed(() => ($mapController() ? $linkedLayout() : null)) + /** Something pasted that is not a link to a layout */ + const $layoutInvalid = computed(() => Boolean($layoutLink().trim()) && !$linkedLayout()) + /** Mapping asked for and no layout to map with yet: the build waits for one */ + const $layoutMissing = computed(() => $mapController() && !$linkedLayout()) /** * Re-ranked as the title changes: the title bonus depends on it (7.2.1). * A path the central directory lists twice is one exe on disk, so it is @@ -123,19 +139,28 @@ export function Game$() { let filledFrom: File | null = null /** * A new archive brings the title and the exe it suggests; without one the - * exe goes with it, while the rest of the form is kept as it was. + * exe goes with it, while the rest of the form is kept as it was. The + * mapping goes once the archive it was set up for does, and not before: the + * first fill may come as late as the box is ticked, when the box is the + * first thing to show the title. */ const fill = () => { const archive = $archive() const zip = $zip() + const previous = filledFrom - if (archive === filledFrom) { + if (archive === previous) { return } filledFrom = archive batch(() => { + if (previous) { + $mapController(false) + $layoutLink('') + } + if (archive && zip) { const name = guessTitle(archive.name, zip) @@ -163,6 +188,11 @@ export function Game$() { $prefixArch, $userData, $overrideWrappers, + $mapController, + $layoutLink, + $layoutId, + $layoutInvalid, + $layoutMissing, $candidates, $selected, $wrappers, diff --git a/packages/frontend/app/uikit/TextInput/TextInput.module.css b/packages/frontend/app/uikit/TextInput/TextInput.module.css index dfc8f96..a8a8c30 100644 --- a/packages/frontend/app/uikit/TextInput/TextInput.module.css +++ b/packages/frontend/app/uikit/TextInput/TextInput.module.css @@ -17,3 +17,8 @@ .root:is(:hover, :focus, :active):not(:disabled) { border-color: var(--colorBrandLight); } + +/* What is typed is wrong, and stays marked while it is corrected. */ +.root[aria-invalid=true]:not(:disabled) { + border-color: var(--colorDanger); +} diff --git a/packages/installer/README.md b/packages/installer/README.md index ecfc4f8..069cc41 100644 --- a/packages/installer/README.md +++ b/packages/installer/README.md @@ -52,7 +52,7 @@ DEST=~/Games PROTON=proton_10 ./nfsmw.run # without dialogs ./nfsmw.run --dry-run # show what would happen, change nothing ``` -The log is `/install.log`, or `/tmp/-install.log` when the game folder cannot be written to. `setup.sh` writes it as the install goes, with the tracebacks, so it is there after a crash as well. The exit codes are described at the top of `install.py`; 4 means "the game is in the library, but not all of its setup went through", and 6 that the progress window was closed before Steam was touched. +The log is `/install.log`, or `/tmp/-install.log` when the game folder cannot be written to. `setup.sh` writes it as the install goes, with the tracebacks, so it is there after a crash as well. The exit codes are described at the top of `install.py`; 4 means "the game is in the library, but not all of its setup went through", and 6 that the progress window was closed before Steam was touched. A `NOTE:` line in the log is something that did not go as planned without making the game any less installed, such as a controller layout that was not set: the last window shows it. Uninstall with what the installer put next to the game: diff --git a/packages/installer/src/install.py b/packages/installer/src/install.py index 2435561..c9207d9 100644 --- a/packages/installer/src/install.py +++ b/packages/installer/src/install.py @@ -1,7 +1,7 @@ #!/usr/bin/env python3 """The Python side of the install in one pass: unpacks the game, shuts the -client down, writes both VDFs, lays out the artwork, brings the client back -and verifies that the edits survived. +client down, writes both VDFs, lays out the artwork, points the game at its +controller layout, brings the client back and verifies that the edits survived. install.py --recipe R --game-dir D --archive Z --proton NAME [--art A] [--dry-run] install.py --recipe R --game-dir D --uninstall @@ -41,6 +41,8 @@ CANCELLED = 6 PROGRESS_REF = None QDBUS = shutil.which('qdbus6') or shutil.which('qdbus') +# What the last window is to say on top of the result, said once each +NOTES = [] # True while the client is down because of us. Every way out of main(), an # exception included, brings it back: left down, Steam looks to the player as # if the installer broke it. @@ -51,6 +53,14 @@ def log(msg=''): print(msg, flush=True) +def note(text): + """A line for the last window: something that did not go as planned + without making the game any less installed. setup.sh shows every one.""" + if text not in NOTES: + NOTES.append(text) + log('NOTE: %s' % text) + + def progress(pct=None, text=None): """Drives the progress bar opened by the shell wrapper. Silently does nothing when running without a GUI - the install does not depend on it.""" @@ -179,7 +189,7 @@ def main(argv=None): ap.add_argument('--steam-user', default='', help='userdata folder to install into, when there is a choice') ap.add_argument('--purge', action='store_true', - help='on uninstall, also remove the game folder and the prefix') + help='on uninstall, also remove the game folder, the prefix and the shader cache') ap.add_argument('--progress-ref', default='', help='DBus reference to the progress bar opened by the shell') a, _ = ap.parse_known_args(argv) @@ -205,6 +215,7 @@ def main(argv=None): return 0 global PROGRESS_REF PROGRESS_REF = a.progress_ref or None + NOTES.clear() log('Plumber %s log' % ('uninstall' if a.uninstall else 'install')) log('%s, %s\n' % (time.strftime('%Y-%m-%d %H:%M:%S'), system_line())) @@ -310,6 +321,7 @@ def run(a): break entry = si.build_entry(appid, name, exe, icon=icon, launch_options=lopts) + layout = recipe.get('controller_layout') if a.dry_run: log('\n--- dry run, nothing is modified ---') @@ -318,6 +330,7 @@ def run(a): log(' %-20s %r' % (k, v)) log('CompatToolMapping[%d] = %s' % (appid, a.proton or '(not set)')) log('artwork from: %s' % (a.art or '(none)')) + log('controller layout: %s' % (layout or '(none)')) log('files: %s' % sc_path) log(' %s' % cfg_path) return 0 @@ -334,6 +347,21 @@ def run(a): log('proton: none chosen, it has to be picked in the game\'s Properties') problems.append('Proton') + # Asked while the client is still up: the only request to Steam the + # install makes, and the client should not be kept down waiting on it + controller = None + if layout: + progress(81, 'Asking Steam about the controller layout...') + try: + controller = si.layout_controller(si.layout_details(layout)) + log('controller layout: %s, made for controller_%s' % (layout, controller)) + except si.LayoutError as exc: + log('controller layout: %s, not set: %s%s' % ( + layout, exc, ' (%s)' % exc.__cause__ if exc.__cause__ else '')) + note( + 'The controller layout was not set: %s. Pick one in the game\'s ' + 'controller settings.' % exc) + # The last point where a closed window can stop the install with nothing # of the person's changed: past it Steam is down and comes back with the game if cancelled(): @@ -350,6 +378,8 @@ def run(a): progress(85, 'Backing up Steam settings...') sc_backup = si.backup(sc_path, log=log) si.backup(cfg_path, log=log) + if controller: + si.backup(si.configset_path(uid, controller), log=log) for attempt in (1, 2, 3): # Another attempt restarts Steam once more, and with the window closed @@ -382,6 +412,14 @@ def run(a): progress(91, 'Installing artwork...') if not step('artwork', si.install_art, uid, appid, a.art, log=log): failed.append('artwork') + layout_set = False + if controller: + progress(92, 'Setting the controller layout...') + layout_set = step('controller layout', si.set_layout, uid, name, controller, layout, log=log) + if not layout_set: + note( + 'The controller layout was not set: its file could not be written. ' + 'Pick one in the game\'s controller settings.') mtime = os.stat(sc_path).st_mtime progress(93, 'Starting Steam again...') @@ -407,6 +445,15 @@ def run(a): log(' Steam never rewrote the file: checking it as it stands') if si.verify(uid, appid, proton, expect_count=expect, log=log): + # Looked at, not retried for: a layout is no reason to restart Steam + if layout_set: + if si.get_layout(uid, name, controller) == layout: + log(' controller layout: kept') + else: + log(' CONTROLLER LAYOUT GONE') + note( + 'Steam did not keep the controller layout. Pick one in the ' + 'game\'s controller settings.') if not step('uninstaller', install_uninstaller, os.path.abspath(a.game_dir), a.recipe, log=log): failed.append('uninstaller') @@ -567,8 +614,10 @@ def do_uninstall(uid, appid, exe_quoted, a, recipe): 'appid=%d' % (idx, found.get('AppName'), appid)) else: log(' no shortcut with this Exe, using computed appid=%d' % appid) + layout = recipe.get('controller_layout') if a.dry_run: - log('dry run: would remove shortcut %d, its CompatToolMapping and artwork' % appid) + log('dry run: would remove shortcut %d, its CompatToolMapping, artwork%s' % ( + appid, ' and controller layout' if layout else '')) return 0 if not stop_steam(): log('Steam did not shut down - aborting') @@ -579,9 +628,14 @@ def do_uninstall(uid, appid, exe_quoted, a, recipe): si.remove_shortcut(uid, appid=appid, exe_quoted=exe_quoted, log=log) si.unset_proton(appid, log=log) si.remove_art(uid, appid, log=log) + if layout: + si.unset_layout(uid, recipe['name'], layout, log=log) if a.purge: pfx = os.path.join(si.STEAM_ROOT, 'steamapps', 'compatdata', str(appid)) + # Steam keeps the shaders it built for the game under its appid, and + # nothing removes them once the shortcut is gone + cache = os.path.join(si.STEAM_ROOT, 'steamapps', 'shadercache', str(appid)) # portable games keep their saves next to the exe - warn about it saves = [ os.path.join(r, f) for r, _, fs in os.walk(gd) @@ -592,10 +646,17 @@ def do_uninstall(uid, appid, exe_quoted, a, recipe): for extra in (os.path.join(gd, '.plumber'),): if os.path.isdir(extra): shutil.rmtree(extra, ignore_errors=True) - for path, what in ((gd, 'game folder'), (pfx, 'prefix')): + for path, what in ((gd, 'game folder'), (pfx, 'prefix'), (cache, 'shader cache')): if os.path.isdir(path): shutil.rmtree(path, ignore_errors=True) log(' removed: %s (%s)' % (what, path)) + # setup.sh made for the game folder alone. Emptied, it goes too; + # anything put there since keeps it + try: + os.rmdir(os.path.dirname(gd)) + log(' removed: empty folder (%s)' % os.path.dirname(gd)) + except OSError: + pass start_steam() if not a.purge: diff --git a/packages/installer/src/selftest.py b/packages/installer/src/selftest.py index ceeb6ae..001e102 100644 --- a/packages/installer/src/selftest.py +++ b/packages/installer/src/selftest.py @@ -24,6 +24,8 @@ import sys import tempfile import time +import urllib.error +import urllib.request import zipfile import zlib @@ -231,6 +233,18 @@ def test_text_keyvalues(): eq(vdf.find_path(parsed, 'nope'), None, 'missing key gives None') +@case +def test_text_keyvalues_written(): + """vdf: text KeyValues come out the way Steam writes them and read back the same""" + data = {'controller_config': {'deus ex': {'workshop': '3036980641'}, 'x\\y': {}}} + text = vdf.text_dumps(data) + + eq(text.splitlines()[:6], [ + '"controller_config"', '{', '\t"deus ex"', '\t{', '\t\t"workshop"\t\t"3036980641"', '\t}'], + 'the layout') + eq(vdf.text_loads(text), data, 'read back, the backslashes as they were') + + # ------------------------------------------------------------ config surgery @case @@ -649,6 +663,77 @@ def test_artwork_names(): eq(si.remove_art(uid, appid, log=quiet), 5, 'all five removed') +@case +def test_layout_key(): + """layout: a non-Steam game is keyed by its name in lower case, without what a file name cannot hold""" + eq(si.layout_key('Deus Ex: Game of the Year Edition'), 'deus ex game of the year edition', 'colon') + eq(si.layout_key('S.T.A.L.K.E.R. Clear Sky'), 'stalker clear sky', 'dots') + + +def layout_record(result=1, app='241100', tags=('feature_mouse', 'controller_neptune')): + """The Workshop's record of a layout, as GetPublishedFileDetails gives it.""" + return { + 'publishedfileid': '3036980641', 'result': result, 'consumer_app_id': app, + 'tags': [{'tag': tag} for tag in tags]} + + +@case +def test_layout_controller_from_its_tags(): + """layout: the controller comes from the Workshop tags, and anything else is refused with a reason""" + eq(si.layout_controller(layout_record()), 'neptune', 'the tag') + for record, why in ( + (layout_record(result=9), 'no public layout'), + (layout_record(app='440'), 'not a controller layout'), + (layout_record(tags=('feature_mouse',)), 'does not say which controller')): + try: + si.layout_controller(record) + except si.LayoutError as exc: + ok(why in str(exc), 'the reason: %s' % exc) + else: + raise AssertionError('took %r' % record) + + +@case +def test_layout_details_without_steam(): + """layout: a Steam that cannot be reached is a reason for the window, not a traceback""" + def offline(*_args, **_kwargs): + raise urllib.error.URLError('Name or service not known') + + saved = urllib.request.urlopen + urllib.request.urlopen = offline + try: + si.layout_details('3036980641') + except si.LayoutError as exc: + eq(str(exc), 'Steam could not be asked which controller it is made for', 'the reason') + else: + raise AssertionError('no error without Steam') + finally: + urllib.request.urlopen = saved + + +@case +def test_layout_set_and_taken_out(): + """layout: set next to other games' in the controller's file, and taken out alone""" + with tempfile.TemporaryDirectory() as tmp: + fake_steam(tmp) + si.set_layout('12345678', 'Hot Pursuit 2', 'neptune', '111', log=quiet) + si.set_layout('12345678', 'Deus Ex: GOTY', 'neptune', '3036980641', log=quiet) + path = si.configset_path('12345678', 'neptune') + games = vdf.text_load(path)['controller_config'] + + eq( + games, + {'hot pursuit 2': {'workshop': '111'}, 'deus ex goty': {'workshop': '3036980641'}}, + 'both games in the file') + eq(si.get_layout('12345678', 'Deus Ex: GOTY', 'neptune'), '3036980641', 'read back') + eq(si.unset_layout('12345678', 'Deus Ex: GOTY', '222', log=quiet), 0, 'a layout picked since stays') + eq(si.unset_layout('12345678', 'Deus Ex: GOTY', '3036980641', log=quiet), 1, 'ours goes') + eq( + vdf.text_load(path)['controller_config'], + {'hot pursuit 2': {'workshop': '111'}}, + 'the other game kept') + + @case def test_artwork_is_dated_now(): """artwork: the files are dated now, not 1970 like everything in meta""" @@ -1163,13 +1248,14 @@ def wait_rewrite(self, path, mtime_before, timeout=210, log=print, tick=None): def run_install(tmp, proton='proton_9', config=CONFIG_WITH_CTM, broken=None, - archive=None, steam=None, entries=None, extra=()): + archive=None, steam=None, entries=None, extra=(), recipe=None): """install.main() on a fake Steam in `tmp`. -> (exit code, log, FakeSteam). The game comes out of `archive`, by default a Zip64 .run holding speed.exe. `broken` names a function of steam_install to replace with one that raises. `steam` is the FakeSteam to run against, `entries` the shortcuts there - before the install, and `extra` more (module, name, value) to patch in. + before the install, `extra` more (module, name, value) to patch in, and + `recipe` more fields for the recipe. """ root = fake_steam(tmp) with open(os.path.join(root, 'config', 'config.vdf'), 'w') as f: @@ -1184,9 +1270,10 @@ def run_install(tmp, proton='proton_9', config=CONFIG_WITH_CTM, broken=None, for name in ('portrait.png', 'icon.png'): with open(os.path.join(art, name), 'wb') as f: f.write(b'\x89PNG') + fields = dict({'name': 'NFS', 'slug': 'nfs', 'exe': 'speed.exe'}, **(recipe or {})) recipe = os.path.join(tmp, 'recipe.json') with open(recipe, 'w') as f: - json.dump({'name': 'NFS', 'slug': 'nfs', 'exe': 'speed.exe'}, f) + json.dump(fields, f) def explode(*_args, **_kwargs): raise RuntimeError('%s blew up' % broken) @@ -1214,17 +1301,19 @@ def explode(*_args, **_kwargs): return rc, out.getvalue(), fake -def run_uninstall(tmp, game, purge=False, entries=None, config=CONFIG_WITH_CTM): +def run_uninstall(tmp, game, purge=False, entries=None, config=CONFIG_WITH_CTM, recipe=None): """install.main() --uninstall on a fake Steam in `tmp`, for the game NFS - (slug nfs) in `game`. -> (exit code, log, FakeSteam)""" + (slug nfs) in `game`, with `recipe` more fields for its recipe. + -> (exit code, log, FakeSteam)""" root = fake_steam(tmp) with open(os.path.join(root, 'config', 'config.vdf'), 'w') as f: f.write(config) if entries is not None: vdf.write_shortcuts(os.path.join(si.user_config('12345678'), 'shortcuts.vdf'), entries) + fields = dict({'name': 'NFS', 'slug': 'nfs', 'exe': 'speed.exe'}, **(recipe or {})) recipe = os.path.join(tmp, 'recipe.json') with open(recipe, 'w') as f: - json.dump({'name': 'NFS', 'slug': 'nfs', 'exe': 'speed.exe'}, f) + json.dump(fields, f) fake = FakeSteam() patches = [ @@ -1416,6 +1505,39 @@ def test_closed_window_gets_no_other_attempt(): eq([e['AppName'] for e in data.values()], ['Hot Pursuit 2'], 'the list as it was') +@case +def test_install_sets_the_layout(): + """install: the layout goes into the file of the controller it is made for""" + with tempfile.TemporaryDirectory() as tmp: + asked = [] + rc, out, _ = run_install(tmp, recipe={'controller_layout': '3036980641'}, extra=[ + (si, 'layout_details', lambda file_id: asked.append(file_id) or layout_record())]) + + eq(rc, 0, 'exit code\n%s' % out) + eq(asked, ['3036980641'], 'Steam asked once, about this layout') + eq(si.get_layout('12345678', 'NFS', 'neptune'), '3036980641', 'the layout for the game') + ok('NOTE:' not in out, 'nothing to say about it:\n%s' % out) + + +@case +def test_install_without_steam_to_ask(): + """install: with Steam out of reach the game goes in, and the window says the layout was not set""" + with tempfile.TemporaryDirectory() as tmp: + def offline(_file_id): + raise si.LayoutError('Steam could not be asked which controller it is made for') + + rc, out, fake = run_install(tmp, recipe={'controller_layout': '3036980641'}, extra=[ + (si, 'layout_details', offline)]) + + eq(rc, 0, 'exit code\n%s' % out) + ok( + 'NOTE: The controller layout was not set: Steam could not be asked which controller ' + 'it is made for. Pick one in the game\'s controller settings.' in out, + 'the line for the window:\n%s' % out) + eq(fake.calls, ['shutdown', 'start'], 'the rest as usual') + ok(not os.path.isdir(si.controller_configs('12345678')), 'no layout file touched') + + @case def test_crash_brings_steam_back(): """install: an unexpected error is logged and Steam is started again""" @@ -1495,6 +1617,29 @@ def test_uninstall_without_an_appid_takes_nothing_else(): 'the Steam Play default') +@case +def test_uninstall_takes_the_layout_out(): + """uninstall: the layout the install set goes, another game's stays, the file backed up""" + with tempfile.TemporaryDirectory() as tmp: + fake_steam(tmp) + si.set_layout('12345678', 'NFS', 'neptune', '3036980641', log=quiet) + si.set_layout('12345678', 'Hot Pursuit 2', 'neptune', '111', log=quiet) + game = os.path.join(tmp, 'Games', 'nfs', 'game') + os.makedirs(game) + + rc, out, _ = run_uninstall(tmp, game, recipe={'controller_layout': '3036980641'}) + + eq(rc, 0, 'exit code\n%s' % out) + eq(si.get_layout('12345678', 'NFS', 'neptune'), None, 'ours gone') + eq(si.get_layout('12345678', 'Hot Pursuit 2', 'neptune'), '111', 'the other kept') + folder = si.controller_configs('12345678') + backups = [f for f in os.listdir(folder) if f.startswith('configset_controller_neptune.vdf.bak-')] + eq(len(backups), 1, 'backups of the file\n%s' % out) + eq( + vdf.text_load(os.path.join(folder, backups[0]))['controller_config'].get('nfs'), + {'workshop': '3036980641'}, 'the backup, taken before ours went') + + @case def test_purge_deletes_only_the_game_folder(): """uninstall: everything goes from //game only, never elsewhere""" @@ -1527,10 +1672,33 @@ def test_purge_deletes_only_the_game_folder(): os.makedirs(os.path.join(game, '.plumber')) with open(os.path.join(game, 'speed.exe'), 'wb') as f: f.write(b'MZ') + with open(os.path.join(tmp, 'Games', 'nfs', 'mine.txt'), 'w') as f: + f.write('mine') rc, out, _ = run_uninstall(tmp, game, purge=True) eq(rc, 0, 'exit code\n%s' % out) ok(not os.path.exists(game), 'the game folder is gone') - ok(os.path.isdir(os.path.join(tmp, 'Games', 'nfs')), 'and nothing above it') + eq(read(tmp, 'Games', 'nfs', 'mine.txt'), b'mine', 'and nothing above it') + + +@case +def test_purge_leaves_nothing_of_the_game(): + """uninstall: the shader cache and the emptied folder go with the game""" + # Both stayed behind after every purge on a Deck and were deleted by hand + with tempfile.TemporaryDirectory() as tmp: + games = os.path.join(tmp, 'Games') + game = os.path.join(games, 'nfs', 'game') + os.makedirs(os.path.join(game, '.plumber')) + appid = si.make_appid('"%s"' % os.path.join(game, 'speed.exe'), 'NFS') + caches = os.path.join(tmp, 'steam', 'steamapps', 'shadercache') + for app in (str(appid), '2229870'): + os.makedirs(os.path.join(caches, app, 'fozpipelinesv6')) + + rc, out, _ = run_uninstall(tmp, game, purge=True) + + eq(rc, 0, 'exit code\n%s' % out) + eq(os.listdir(caches), ['2229870'], "the shader caches left, only the game's gone") + ok(not os.path.exists(os.path.join(games, 'nfs')), 'the emptied folder is gone') + ok(os.path.isdir(games), 'the folder above it is kept') def sh(script, *args): diff --git a/packages/installer/src/setup.sh b/packages/installer/src/setup.sh index 2f8f432..3be7bf2 100755 --- a/packages/installer/src/setup.sh +++ b/packages/installer/src/setup.sh @@ -237,13 +237,18 @@ progress_pct 100 progress_close if [ -z "$DRYRUN" ] && [ -n "$GUI" ] && have kdialog; then + # install.py writes "NOTE: ..." lines for the person: something that did + # not go as planned without making the game any less installed + _notes=$(grep '^NOTE: ' "$LOG" 2>/dev/null | cut -c7- || true) case $RC in 0) kdialog --title "Done" --icon dialog-ok \ --msgbox "$NAME is installed. It is now in your Steam library. -If you don't see it, switch library pages to refresh." 2>/dev/null || true +If you don't see it, switch library pages to refresh.${_notes:+ + +$_notes}" 2>/dev/null || true ;; 4) # The game is in the library, but a step after the shortcut did not @@ -253,7 +258,9 @@ If you don't see it, switch library pages to refresh." 2>/dev/null || true If the game does not start, open its Properties, then Compatibility, turn on \"Force the use of a specific Steam Play compatibility tool\" -and pick a Proton version. +and pick a Proton version.${_notes:+ + +$_notes} Details in: $LOG" 2>/dev/null || true diff --git a/packages/installer/src/steam_install.py b/packages/installer/src/steam_install.py index a96f07b..4be5725 100644 --- a/packages/installer/src/steam_install.py +++ b/packages/installer/src/steam_install.py @@ -1,14 +1,18 @@ -"""The Steam side of the install: user, shortcut, Proton, artwork. +"""The Steam side of the install: user, shortcut, Proton, artwork, controller +layout. Everything that needs an understanding of Steam's formats. The shell wrapper (setup.sh) handles the questions and the extraction and calls into here. """ +import json import os import re import shutil import sys import time +import urllib.parse +import urllib.request import zlib sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) @@ -474,6 +478,128 @@ def remove_art(uid, appid, log=print): return n +# -------------------------------------------------------- controller layout +# A layout from the Workshop, for a game made for keyboard and mouse (7.8). +# Which layout a game uses is kept per account, in a file per controller type; +# the client fetches the layout itself once a game points at its Workshop ID. + +LAYOUT_DETAILS_URL = ( + 'https://api.steampowered.com/ISteamRemoteStorage/GetPublishedFileDetails/v1/') +# The app the Workshop keeps controller layouts under +CONTROLLER_CONFIGS_APP = '241100' + + +class LayoutError(RuntimeError): + """A layout that cannot be set, with the reason in words for the person.""" + + +def controller_configs(uid): + return os.path.join( + STEAM_ROOT, 'steamapps', 'common', 'Steam Controller Configs', uid, 'config') + + +def configset_path(uid, controller): + """The file that says which layout each game uses with one controller.""" + return os.path.join( + controller_configs(uid), 'configset_controller_%s.vdf' % controller) + + +def layout_key(name): + """What the client files a non-Steam game's layout under: its name in lower + case, without what a file name cannot hold. Steam ROM Manager keys the + shortcuts it makes the same way.""" + return re.sub(r'[/\\?%*:|"<>.]', '', name.lower()) + + +def layout_details(file_id, timeout=15): + """The Workshop's record of a layout. -> dict + + The site cannot ask for it, since Steam's answer carries no CORS headers, + but this is not a browser, and the request needs no key.""" + body = urllib.parse.urlencode( + {'itemcount': 1, 'publishedfileids[0]': file_id}).encode() + try: + with urllib.request.urlopen(LAYOUT_DETAILS_URL, data=body, timeout=timeout) as reply: + answer = json.load(reply) + except (OSError, ValueError) as exc: + raise LayoutError( + 'Steam could not be asked which controller it is made for') from exc + try: + return answer['response']['publishedfiledetails'][0] + except (KeyError, IndexError, TypeError) as exc: + raise LayoutError('Steam answered about something else') from exc + + +def layout_controller(details): + """The controller a layout is made for, from its Workshop tags. + -> 'neptune', 'xboxone', 'ps5', ...""" + if str(details.get('result')) != '1': + raise LayoutError('the Workshop has no public layout with this ID') + if str(details.get('consumer_app_id')) != CONTROLLER_CONFIGS_APP: + raise LayoutError('this Workshop item is not a controller layout') + for tag in details.get('tags') or []: + name = tag.get('tag', '') if isinstance(tag, dict) else '' + if name.startswith('controller_'): + return name[len('controller_'):] + raise LayoutError('the layout does not say which controller it is made for') + + +def _games(data): + """The block of games in a configset file, made if it is not there.""" + hit = next((k for k in data if k.lower() == 'controller_config'), None) + if hit is None or not isinstance(data[hit], dict): + hit = hit or 'controller_config' + data[hit] = {} + return data[hit] + + +def set_layout(uid, name, controller, file_id, log=print): + """Points the game at a Workshop layout for one controller. Steam must be + down; it fetches the layout itself.""" + path = configset_path(uid, controller) + data = vdf.text_load(path) if os.path.isfile(path) else {} + _games(data)[layout_key(name)] = {'workshop': file_id} + os.makedirs(os.path.dirname(path), exist_ok=True) + vdf.replace_file(path, vdf.text_dumps(data).encode('utf-8')) + log(' controller layout: %s for controller_%s' % (file_id, controller)) + + +def get_layout(uid, name, controller): + """-> the Workshop ID the game's layout for the controller points at, or None""" + path = configset_path(uid, controller) + if not os.path.isfile(path): + return None + entry = vdf.find_path(vdf.text_load(path), 'controller_config', layout_key(name)) + return entry.get('workshop') if isinstance(entry, dict) else None + + +def unset_layout(uid, name, file_id, log=print): + """Takes the game out of every controller's file where it still points at + the layout the install set, and leaves a layout picked since alone. A file + is backed up before it is rewritten, as the install does it: which files + those are is known only here. + -> how many files it was taken out of""" + folder = controller_configs(uid) + if not os.path.isdir(folder): + return 0 + key = layout_key(name) + taken = 0 + for f in sorted(os.listdir(folder)): + if not (f.startswith('configset_controller_') and f.endswith('.vdf')): + continue + path = os.path.join(folder, f) + data = vdf.text_load(path) + games = vdf.find_path(data, 'controller_config') + entry = games.get(key) if isinstance(games, dict) else None + if isinstance(entry, dict) and entry.get('workshop') == file_id: + del games[key] + backup(path, log=log) + vdf.replace_file(path, vdf.text_dumps(data).encode('utf-8')) + log(' controller layout: removed from %s' % f) + taken += 1 + return taken + + # ---------------------------------------------------------------- verifying def verify(uid, appid, expect_tool, expect_count=None, log=print): diff --git a/packages/installer/src/vdf.py b/packages/installer/src/vdf.py index 3dcf895..1eab5ef 100644 --- a/packages/installer/src/vdf.py +++ b/packages/installer/src/vdf.py @@ -134,6 +134,21 @@ def text_load(path): return text_loads(f.read()) +def text_dumps(node, depth=0): + """KeyValues from nested dicts, laid out the way Steam writes them: a tab + a level, two between a key and its value. Strings go out as text_loads + left them, every backslash included.""" + tabs = '\t' * depth + out = [] + for key, value in node.items(): + if isinstance(value, dict): + out.append('%s"%s"\n%s{\n%s%s}\n' % ( + tabs, key, tabs, text_dumps(value, depth + 1), tabs)) + else: + out.append('%s"%s"\t\t"%s"\n' % (tabs, key, value)) + return ''.join(out) + + def find_path(node, *keys): """Case-insensitive descent through nested keys. -> value or None.""" cur = node