Skip to content

Add Verse language support - #5555

Open
Simon GAY (simnJS) wants to merge 1 commit into
microsoft:mainfrom
simnJS:add-verse
Open

Simon GAY (simnJS) wants to merge 1 commit into
microsoft:mainfrom
simnJS:add-verse

Conversation

@simnJS

Copy link
Copy Markdown

Adds a Monarch tokenizer for Verse, the programming language Epic Games created for Unreal Editor for Fortnite (UEFN), for *.verse files.

Relevance

Changes

  • src/languages/definitions/verse/: register.ts, verse.ts (tokenizer and language configuration) and verse.test.ts.
  • src/languages/definitions/register.all.ts: registers the language.
  • website/src/website/data/home-samples/sample.verse.txt: sample from simnJS/verse-grammar, plus an indented comment.

The token rules follow the Verse grammar that Epic Games publishes under CC0 in verselang/book, like the Verse lexers I contributed to Pygments (pygments/pygments#3422) and Chroma.

Tokens

  • Comments: # line comments, nested <# ... #> block comments, and <#> indented comments. An indented comment covers the following lines that are indented deeper than the <#> line; the tokenizer keeps that indentation in the state name and matches it with $S2.
  • Block comments inside strings are comments, since Verse removes them from the string value ("ab<#x#>cd" = "abcd").
  • Strings with {...} interpolation (nested braces, strings inside the interpolation, interpolation that continues on the next line), the escapes \t \n \r \" \' \\ \{ \} \< \> \& \# \~, invalid escapes and unterminated strings.
  • Characters: 'a', '\n', '😀', 0o61, 0u1F600.
  • Numbers: decimal, hex, floats with exponent, suffixes such as 12.34f64, and ranges like 0..9.
  • Specifiers (<public>, <decides>, <scoped{...}>, <getter(...)>) and attributes (@editable) as annotation. Comparisons such as A<B, B>C stay operators.
  • Keywords, built-in types (keyword.type), operators, and paths in using { /Verse.org/Simulation }, import(...) and (/Verse.org/Random:) qualifiers (constant.path).

The language configuration sets # and <# #> for comment toggling, indents after a line ending in : or =, and uses off-side folding.

Testing

  • npm test: all tokenization tests pass, including the new verse cases (each comment form, interpolation with nested braces, a comment inside a string, specifiers next to identifiers, comparisons that are not specifiers, paths), and the sample check passes.
  • npm run build, npm run compile --prefix webpack-plugin and npm run package-for-smoketest pass; Verse is in the webpack, esbuild and Vite bundles. I did not run the Playwright smoke tests locally.
  • Checked the sample in a local page built from out/monaco-editor, with the vs and vs-dark themes: Enter after a line ending in : or = indents, and the comment commands insert # and <# ... #>.

Adds a Monarch tokenizer, language configuration, tokenization tests
and a website sample for Verse (*.verse), the programming language of
Unreal Editor for Fortnite.
@simnJS

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant