diff --git a/docs/templating.md b/docs/templating.md index 89be66a3285..cd93f62ed6d 100644 --- a/docs/templating.md +++ b/docs/templating.md @@ -50,13 +50,41 @@ java -cp /path/totemplate-classpath-example-1.0-SNAPSHOT.jar:modules/openapi-gen Note that our template directory is relative to the resource directory of the JAR defined on the classpath. +### Retrieving Templates + +You will need to find and retrieve the templates for your desired generator in order to redefine structures, documentation, or API logic. We cover template customization in the following sections. + +In OpenAPI Generator 5.0 and later, you can use the CLI command `author template` to extract embedded templates for your target generator. For example: + +``` +openapi-generator author template -g java --library webclient +``` + +For OpenAPI Generator versions prior to 5.0, you will want to find the [resources directory](https://github.com/OpenAPITools/openapi-generator/tree/master/modules/openapi-generator/src/main/resources) for the generator you want to extend. This is generally easy to find as directories commonly follow the convention of `resources/`. In cases where you're unsure, you will need to find the `embeddedTemplateDir` assignment in your desired generator. This is almost always assigned in the constructor of the generator class. The C# .Net Core generator assigns this as: + +``` +embeddedTemplateDir = templateDir = "csharp-netcore"; +``` + +These templates are in our source repository at [modules/openapi-generator/src/main/resources/csharp-netcore](https://github.com/OpenAPITools/openapi-generator/tree/master/modules/openapi-generator/src/main/resources/csharp-netcore). Be sure to select the tag or branch for the version of OpenAPI Generator you're using before grabbing the templates. + +**NOTE** If you have specific logic you'd like to modify such as modifying the generated README, you _only_ need to pull and modify this individual template. OpenAPI Generator will lookup templates in this order: + +* User customized library path (e.g. `custom_template/libraries/feign/model.mustache`) +* User customized generator top-level path (e.g. `custom_template/model.mustache`) +* Embedded library path (e.g. `resources/Java/libraries/feign/model.mustache`) +* Embedded top-level path (e.g. `resources/Java/model.mustache`) +* Common embedded path (e.g. `resources/_common/model.mustache`) + ### Custom Logic For this example, let's modify a Java client to use AOP via [jcabi/jcabi-aspects](https://github.com/jcabi/jcabi-aspects). We'll log API method execution at the `INFO` level. The jcabi-aspects project could also be used to implement method retries on failures; this would be a great exercise to further play around with templating. The Java generator supports a `library` option. This option works by defining base templates, then applying library-specific template overrides. This allows for template reuse for libraries sharing the same programming language. Templates defined as a library need only modify or extend the templates concerning the library, and generation falls back to the root templates (the "defaults") when not extended by the library. Generators which support the `library` option will only support the libraries known by the generator at compile time, and will throw a runtime error if you try to provide a custom library name. -To get started, we will need to copy our target generator's directory in full. The directory will be located under `modules/opeanpi-generator/src/main/resources/{generator}`. In general, the generator directory matches the generator name (what you would pass to the `generator` option), but this is not a requirement-- if you are having a hard time finding the template directory, look at the `embeddedTemplateDir` option in your target generator's implementation. +To get started, we will need to copy our target generator's directory in full. + +The directory will be located under `modules/opeanpi-generator/src/main/resources/{generator}`. In general, the generator directory matches the generator name (what you would pass to the `generator` option), but this is not a requirement-- if you are having a hard time finding the template directory, look at the `embeddedTemplateDir` option in your target generator's implementation. If you've already cloned openapi-generator, find and copy the `modules/opeanpi-generator/src/main/resources/Java` directory. If you have the [Refined GitHub](https://github.com/sindresorhus/refined-github) Chrome or Firefox Extension, you can navigate to this directory on GitHub and click the "Download" button. Or, to pull the directory from latest master: diff --git a/docs/usage.md b/docs/usage.md index cddfd145ca9..4522d551009 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -15,13 +15,14 @@ openapi-generator help usage: openapi-generator-cli [] The most commonly used openapi-generator-cli commands are: + author Utilities for authoring generators or customizing templates. config-help Config help for chosen lang generate Generate code with the specified generator. - help Display help information + help Display help information about openapi-generator list Lists the available generators meta MetaGenerator. Generator for creating a new template set and configuration for Codegen. The output will be based on the language you specify, and includes default templates to include. validate Validate specification - version Show version information + version Show version information used in tooling See 'openapi-generator-cli help ' for more information on a specific command. @@ -670,3 +671,87 @@ EOF openapi-generator batch *.yaml ``` +## author + +This command group contains utilities for authoring generators or customizing templates. + +``` +openapi-generator help author +NAME + openapi-generator-cli author - Utilities for authoring generators or + customizing templates. + +SYNOPSIS + openapi-generator-cli author + openapi-generator-cli author template [(-v | --verbose)] + [(-o | --output )] + [--library ] + (-g | --generator-name ) + +OPTIONS + --help + Display help about the tool + + --version + Display full version output + +COMMANDS + With no arguments, Display help information about openapi-generator + + template + Retrieve templates for local modification + + With --verbose option, verbose mode + + With --output option, where to write the template files (defaults to + 'out') + + With --library option, library template (sub-template) + + With --generator-name option, generator to use (see list command for + list) +``` + +### template + +This command allows user to extract templates from the CLI jar which simplifies customization efforts. + +``` +NAME + openapi-generator-cli author template - Retrieve templates for local + modification + +SYNOPSIS + openapi-generator-cli author template + (-g | --generator-name ) + [--library ] + [(-o | --output )] + [(-v | --verbose)] + +OPTIONS + -g , --generator-name + generator to use (see list command for list) + + --library + library template (sub-template) + + -o , --output + where to write the template files (defaults to 'out') + + -v, --verbose + verbose mode +``` + +Example: + +Extract Java templates, limiting to the `webclient` library. + +``` +openapi-generator author template -g java --library webclient +``` + +Extract all Java templates: + +``` +openapi-generator author template -g java +``` diff --git a/modules/openapi-generator-cli/src/main/java/org/openapitools/codegen/OpenAPIGenerator.java b/modules/openapi-generator-cli/src/main/java/org/openapitools/codegen/OpenAPIGenerator.java index f0e42adb4a9..335b1232de6 100644 --- a/modules/openapi-generator-cli/src/main/java/org/openapitools/codegen/OpenAPIGenerator.java +++ b/modules/openapi-generator-cli/src/main/java/org/openapitools/codegen/OpenAPIGenerator.java @@ -57,6 +57,11 @@ public class OpenAPIGenerator { GenerateBatch.class ); + builder.withGroup("author") + .withDescription("Utilities for authoring generators or customizing templates.") + .withDefaultCommand(HelpCommand.class) + .withCommands(AuthorTemplate.class); + try { builder.build().parse(args).run(); diff --git a/modules/openapi-generator-cli/src/main/java/org/openapitools/codegen/cmd/AuthorTemplate.java b/modules/openapi-generator-cli/src/main/java/org/openapitools/codegen/cmd/AuthorTemplate.java new file mode 100644 index 00000000000..50b6b4a2815 --- /dev/null +++ b/modules/openapi-generator-cli/src/main/java/org/openapitools/codegen/cmd/AuthorTemplate.java @@ -0,0 +1,168 @@ +package org.openapitools.codegen.cmd; + +import io.airlift.airline.Command; +import io.airlift.airline.Option; +import org.apache.commons.lang3.StringUtils; +import org.openapitools.codegen.CodegenConfig; +import org.openapitools.codegen.CodegenConfigLoader; +import org.openapitools.codegen.CodegenConstants; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.File; +import java.io.IOException; +import java.net.URI; +import java.net.URISyntaxException; +import java.nio.file.*; +import java.nio.file.spi.FileSystemProvider; +import java.util.*; +import java.util.regex.Pattern; +import java.util.stream.Stream; + +@SuppressWarnings({"FieldMayBeFinal", "FieldCanBeLocal", "unused"}) +@Command(name = "template", description = "Retrieve templates for local modification") +public class AuthorTemplate extends OpenApiGeneratorCommand { + + private static final Logger LOGGER = LoggerFactory.getLogger(AuthorTemplate.class); + + @Option(name = {"-g", "--generator-name"}, title = "generator name", + description = "generator to use (see list command for list)", + required = true) + private String generatorName; + + @Option(name = {"--library"}, title = "library", description = CodegenConstants.LIBRARY_DESC) + private String library; + + @Option(name = {"-o", "--output"}, title = "output directory", + description = "where to write the template files (defaults to 'out')") + private String output = ""; + + @Option(name = {"-v", "--verbose"}, description = "verbose mode") + private boolean verbose; + + private Pattern pattern = null; + + @Override + void execute() { + CodegenConfig config = CodegenConfigLoader.forName(generatorName); + String templateDirectory = config.templateDir(); + + log("Requesting '{}' from embedded resource directory '{}'", generatorName, templateDirectory); + + Path embeddedTemplatePath; + try { + URI uri = Objects.requireNonNull(this.getClass().getClassLoader().getResource(templateDirectory)).toURI(); + + if ("jar".equals(uri.getScheme())) { + Optional provider = FileSystemProvider.installedProviders() + .stream() + .filter(p -> p.getScheme().equalsIgnoreCase("jar")) + .findFirst(); + + if (!provider.isPresent()) { + throw new ProviderNotFoundException("Unable to load jar file system provider"); + } + + try { + provider.get().getFileSystem(uri); + } catch (FileSystemNotFoundException ex) { + // File system wasn't loaded, so create it. + provider.get().newFileSystem(uri, Collections.emptyMap()); + } + } + + embeddedTemplatePath = Paths.get(uri); + + log("Copying from jar location {}", embeddedTemplatePath.toAbsolutePath().toString()); + + File outputDir; + if (StringUtils.isNotEmpty(output)) { + outputDir = new File(output); + } else { + outputDir = new File("out"); + } + + Path outputDirPath = outputDir.toPath(); + if (!Files.exists(outputDirPath)) { + Files.createDirectories(outputDirPath); + } + List generatedFiles = new ArrayList<>(); + try (final Stream templates = Files.walk(embeddedTemplatePath)) { + templates.forEach(template -> { + log("Found template: {}", template.toAbsolutePath()); + Path relativePath = embeddedTemplatePath.relativize(template); + if (shouldCopy(relativePath)) { + Path target = outputDirPath.resolve(relativePath.toString()); + generatedFiles.add(target); + try { + if (Files.isDirectory(template)) { + if (Files.notExists(target)) { + log("Creating directory: {}", target.toAbsolutePath()); + Files.createDirectories(target); + } + } else { + if (target.getParent() != null && Files.notExists(target.getParent())) { + log("Creating directory: {}", target.getParent()); + Files.createDirectories(target.getParent()); + } + log("Copying to: {}", target.toAbsolutePath()); + Files.copy(template, target, StandardCopyOption.REPLACE_EXISTING); + } + } catch (IOException e) { + LOGGER.error("Unable to create target location '{}'.", target); + } + } else { + log("Directory is excluded by library option: {}", relativePath); + } + }); + } + + if (StringUtils.isNotEmpty(library) && !generatedFiles.isEmpty()) { + Path librariesPath = outputDirPath.resolve("libraries"); + Path targetLibrary = librariesPath.resolve(library); + String librariesPrefix = librariesPath.toString(); + if (!Files.isDirectory(targetLibrary)) { + LOGGER.warn("The library '{}' was not extracted. Please verify the spelling and retry.", targetLibrary); + } + generatedFiles.stream() + .filter(p -> p.startsWith(librariesPrefix)) + .forEach(p -> { + if (p.startsWith(targetLibrary)) { + // We don't care about empty directories, and not need to check directory for files. + if (!Files.isDirectory(p)) { + // warn if the file was not written + if (Files.notExists(p)) { + LOGGER.warn("An expected library file was not extracted: {}", p.toAbsolutePath()); + } + } + } else { + LOGGER.warn("The library filter '{}' extracted an unexpected library path: {}", library, p.toAbsolutePath()); + } + }); + } + + LOGGER.info("Extracted templates to '{}' directory. Refer to https://openapi-generator.tech/docs/templating for customization details.", outputDirPath); + } catch (URISyntaxException | IOException e) { + LOGGER.error("Unable to load embedded template directory.", e); + } + } + + private void log(String format, Object... arguments) { + if (verbose) { + LOGGER.info(format, arguments); + } + } + + private boolean shouldCopy(Path relativePath) { + String path = relativePath.toString(); + if (StringUtils.isNotEmpty(library) && path.contains("libraries")) { + if (pattern == null) { + pattern = Pattern.compile(String.format(Locale.ROOT, "libraries[/\\\\]{1}%s[/\\\\]{1}.*", Pattern.quote(library))); + } + + return pattern.matcher(path).matches(); + } + + return true; + } +} diff --git a/modules/openapi-generator-cli/src/test/java/org/openapitools/codegen/cmd/AuthorTemplateTest.java b/modules/openapi-generator-cli/src/test/java/org/openapitools/codegen/cmd/AuthorTemplateTest.java new file mode 100644 index 00000000000..4826464ad91 --- /dev/null +++ b/modules/openapi-generator-cli/src/test/java/org/openapitools/codegen/cmd/AuthorTemplateTest.java @@ -0,0 +1,71 @@ +package org.openapitools.codegen.cmd; + +import io.airlift.airline.Cli; +import org.testng.Assert; +import org.testng.ITestContext; +import org.testng.annotations.BeforeTest; +import org.testng.annotations.Test; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Arrays; + +public class AuthorTemplateTest { + Path outputDirectory; + + @BeforeTest + public void setUp(ITestContext ctx) throws IOException { + outputDirectory = Files.createTempDirectory("AuthorTemplateTest"); + outputDirectory.toFile().deleteOnExit(); + } + + @Test + public void smokeTestAuthorTemplateCommand(){ + Cli.CliBuilder builder = createBuilder(); + String[] arguments = new String[]{ + "author", + "template", + "-g", + "java", + "--library", + "webclient", + "--output", + outputDirectory.toAbsolutePath().toString() + }; + builder.build().parse(arguments).run(); + + // spot check root files + Assert.assertTrue(Files.exists(outputDirectory.resolve("ApiClient.mustache"))); + Assert.assertTrue(Files.exists(outputDirectory.resolve("api_doc.mustache"))); + Assert.assertTrue(Files.exists(outputDirectory.resolve("pom.mustache"))); + + Assert.assertTrue(Files.exists(outputDirectory.resolve("auth/OAuth.mustache"))); + + // check libraries files and subdirectories + Assert.assertTrue(Files.exists(outputDirectory.resolve("libraries/webclient/ApiClient.mustache"))); + Assert.assertTrue(Files.exists(outputDirectory.resolve("libraries/webclient/pom.mustache"))); + Assert.assertTrue(Files.exists(outputDirectory.resolve("libraries/webclient/auth/OAuth.mustache"))); + + // check non-existence of unselected libraries + Assert.assertFalse(Files.exists(outputDirectory.resolve("libraries/feign/build.gradle.mustache"))); + Assert.assertFalse(Files.exists(outputDirectory.resolve("libraries/feign/auth/OAuth.mustache"))); + + Assert.assertFalse(Files.exists(outputDirectory.resolve("libraries/jersey2/api_doc.mustache"))); + Assert.assertFalse(Files.exists(outputDirectory.resolve("libraries/jersey2/auth/HttpBasicAuth.mustache"))); + + Assert.assertFalse(Files.exists(outputDirectory.resolve("libraries/okhttp-gson/api.mustache"))); + Assert.assertFalse(Files.exists(outputDirectory.resolve("libraries/okhttp-gson/auth/RetryingOAuth.mustache"))); + } + + private Cli.CliBuilder createBuilder(){ + Cli.CliBuilder builder = new Cli.CliBuilder<>("openapi-generator-cli"); + + builder.withGroup("author") + .withDescription("Utilities for authoring generators or customizing templates.") + .withDefaultCommand(HelpCommand.class) + .withCommands(AuthorTemplate.class); + + return builder; + } +}