199 Commits

Author SHA1 Message Date
William Cheng
2578ef5e63
Prepare snapshot (v5.0.0) (#7986)
* Revert "prepare 5.0.0-beta3 release"

This reverts commit 3ebe774df37b43348bac1310a272de53a2d7040b.

* revert to snapshot

* update samples
2020-11-20 19:19:59 +08:00
William Cheng
2715f1371a
Prepare 5.0.0 beta3 (#7984)
* prepare 5.0.0-beta3 release

* update samples

* remove beta3 from readme
2020-11-20 15:18:58 +08:00
Guillaume GILL
9236d5699d
Expose batch subcommand (#7608)
Co-authored-by: Guillaume GILL <g.gill@oceasoft.com>
2020-10-05 20:11:30 -04:00
Jim Schubert
d854c89272
[docs] Add clarity around configuration options (#7547) 2020-09-30 20:56:28 -04:00
Jim Schubert
6e937ba9f9
[batch] Option to clean files before generation (#7465) 2020-09-23 08:50:23 -04:00
Jim Schubert
3ca6bc2518
[sonar] Java 11, use -nsu, exclude maven/online projects (#7459) 2020-09-20 16:02:29 -04:00
dtiller
6909c888d2
[cli] Batch: support multiple/nested !include directive (#7354)
Co-authored-by: Jim Schubert <james.schubert@gmail.com>
2020-09-16 22:45:26 -04:00
Jim Schubert
bf0bd29ef1
[core][feature] User custom added templates (#7366) 2020-09-12 18:09:59 -04:00
William Cheng
2fd23f505e
Prepare 5.0.0-SNAPSHOT (#7347)
* Revert "Prepare 5.0.0-beta2 release (#7345)"

This reverts commit 212b4f784742744b7b238607dcefb23ec9db64d7.

* update readme
2020-09-04 15:31:54 +08:00
William Cheng
212b4f7847
Prepare 5.0.0-beta2 release (#7345)
* update version

* update doc

* update samples
2020-09-04 13:03:45 +08:00
Jim Schubert
8eea149e14
[doc] Document usage of post-process file feature (#7315) 2020-08-31 13:44:44 +08:00
William Cheng
f05aa8e80d
Prepare 5.0.0-SNAPSHOT (#6812)
* Revert "Prepare 5.0.0-beta release (#6803)"

This reverts commit a77fd443ad9c5fcd85186b76a15abd1b02887576.

* prepare 5.0.0 snapshot

* update non-java-invalid-spec.xml
2020-06-30 01:19:46 +08:00
William Cheng
a77fd443ad
Prepare 5.0.0-beta release (#6803)
* prepare 5.0.0-beta release

* fix java-client pom
2020-06-29 23:06:47 +08:00
Jim Schubert
0cb080d5f1
[ci][cli] Allow invoking generate-samples.sh with single file + args (#6609) 2020-06-11 23:39:26 +08:00
Jim Schubert
60ceded171
[ci][cli] Moving ensures script to config-based batch generation of samples (#6509)
* Ensure CLI-level generator settings are available in additional properties

* Initial conversion to yaml configs

Configs prefix with 1- need to be manually evaluated.

* Add "other" configs not in ensure-up-to-date

* Add other/openapi3 files

* Cleanup all generation scripts

* Clean up: add missed configs and fix some openapi 3 diffs

* Move generate-samples script, error on batch failures

* Temporarily disable elm which requires skip validation of spec

* CI updates (todo: run all generators through appveyor or move to github workflows)

* Add success count to batch generation command output

* [samples] Regenerate

* Remove bin/windows

* Generate swift5 samples in bitrise

* Expand user input glob pattern

* Regenerate samples

* Update PR template

* Support config based generateAliasAsModel

* [samples] Regenerate

* [rust] Generate as alias for all samples configs

* [csharp] Move test staged file changes to in-directory

* Include "live" test files and ignore test cache files from samples.ci

* Remove concept of samples.ci staging

Generators support .openapi-generator-ignore, allowing maintainers to
explicitly ignore the regeneration of files which have been modified.

Note that the tooling does not overwrite test files whenever those files
exist, and it's not entirely necessary to add test files to the ignore
file.

* Re-add meta generator scripts

* Modify ensure-up-to-date to log stdout, so script does not look like it hangs on slower machines

* [csharp] Regenerate sample

* Set generateAliasAsModel CodegenConfigurator#toContext

This ensures the property (which is effectively a thread-local) gets set
as expected if the caller resets GlobalSettings as is done in the
GenerateBatch command.

* [rust] Regenerate samples

* Re-enable elm

* [java] Support OffsetDateTime example, fail ensures script on any generation error

* [samples] Regenerate

* Fix jersey2-java8 ignore file

* Fix elm spec validation

* Force UTC in sample generation to avoid timezone conflicts during generation (user vs ci)

Co-authored-by: William Cheng <wing328hk@gmail.com>
2020-06-09 18:28:58 +08:00
Jim Schubert
58ed6afc0d
Update Generate.java (#6515)
Removed missed -D messages
2020-06-01 21:08:50 -04:00
Jim Schubert
7e5f720f20
Remove deprecations 5.0 (#6060)
* [breaking] Enforce vendor extension naming convention

* [breaking] Rename system properties to global properties

* [docs] Update site with global properties list and usage explanation

* Use proper vendor extension casing in all templates

* Set remaining vendor extensions to convention of lower kebab-cased with x- prefix

* [samples] Regenerate

* Update modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/RustServerCodegen.java

Before we were adding hasPathParams twice, once with !op.pathParams.isEmpty(), and then again with hasPathParams. This was probably caused by a mistaken merge.

This is causing the difference in samples

Co-authored-by: Richard Whitehouse <richard.whitehouse@metaswitch.com>

* [Samples] Regenerated!

* Fix -D conversion to additional-properties, missed in bat files

* JERSEY2 option changed

* [samples] Regenerate

* [scala][finch] Fix remaining vendor extensions format to conventino

* [scala] The -D option was replaced with --global-property

* [samples] Regenerate

* Fix -DskipFormModel usage which has been moved to --global-property skipFormModel=true

* [samples] Regenerate

Co-authored-by: Richard Whitehouse <richard.whitehouse@metaswitch.com>
2020-05-31 19:14:31 +08:00
Jim Schubert
a47e522fae
[core] Refactor templating management (#6357)
* [core] Refactor templating management

This refactors template management to get logic out of DefaultGenerator
and to provide a cleaner API to template search and read/compile.

Deprecates MockDefaultGenerator, which is not a mock and causes
in-memory retention of file contents. Maintainers should prefer
executing a "dryRun" with new DefaultGenerator(true) or do true
mock/spies if evaluating template intermediaries is truly necessary.
Tests may read written files with lower overhead than the in-process
retention of those bytes.

This attempts to maintain some compatibility with existing templating
adapter interfaces. Any breaking change here would be unintentional but
minimal effort to retarget the new interface.

* Tests for dry run file outputs

* Update API usage in Meta, test TemplateManager

* Wait on lastModified, lookup by filename in SpringCodegenTest

* Test DefaultGenerator + ignore file

* Move config.processOpenAPI in DefaultGenerator

* Fix wrong use of libraries templateDirector (java)

The samples scripts for Java incorrectly referenced the libraries
directories directly rather than the upper-level Java directory. This
was incorrect usage of template directories, because the generator
expects to be given the "language" directory and perform a lookup for
missing templates in the order:

* user defined libraries directory
* user defined language root
* embedded libraries directory
* embedded language root
* _common directory

This is incorrect in our samples scripts because a user or maintainer
has the expectation that any template change to files at the Java/ root
should also be honored on generation if the script specifies a custom
template directory.

* Fix handlebars extension usage, clean up Meta tasks

HandlebarseEngineAdapter previously didn't handle files without
extensions in the same was as the MustacheEngineAdapter. This now allows
for files without extension (or dotfiles) to lookup in the same
location.

Meta tasks are cleaned up to use template manager only, rather than
attempting to create an "empty" generator to use the previous templating
specific methods.

* Update kotlin-multiplatform gradle wrapper

* Rename GraphQL .gitignore template

The .gitignore file is unable to load via classpath resource from the
graphql node server resource directory (for unknown reasons). Before
this change, the missing template would fail silently.

A .gitignore file may exist in other directories and load as expected.
Added a default .gitignore to _common as a fallback so as not to break
any custom generators which may also be failing silently.

* Log entire stacktrace in go sdk built by gradle in AppVeyor

* Rename PHP .gitignore to gitignore

Java resources may not load .gitignore, this follows suit with other
generators and uses "gitignore" (some use "gitignore.mustache").

* [php] Rename .gitignore templates to gitignore

* Use same classpath lookup in common locator

* [rust] Properly escape empty triple-braces

* [samples] Regenerate
2020-05-30 00:19:03 -04:00
Jim Schubert
7d4bbcc29b
[cli] new 'author template' command (#6441)
* [cli] new 'author template' command

This new command allows users to extract templates for authoring
(customization) without the complexity of finding and downloading a
specific directory for their versioned artifact.

Example usage:

```
openapi-generator author template -g java --library webclient
```

This will write all templates with library-specific templates to the
'./out' directory relative to the current directory.

CLI will refer the user to
https://openapi-generator.tech/docs/templating after generation

* [docs] Usage of author template command

* Log warning if author template fails to output requested library
2020-05-28 00:12:59 -04:00
William Cheng
1d606499ee Merge remote-tracking branch 'origin/5.0.x' 2020-05-06 22:51:05 +08:00
William Cheng
003165c2c2
Prepare 4.3.1 release (#6187)
* prepare 4.3.1 release

* update doc
2020-05-06 17:10:31 +08:00
Jim Schubert
cc623ba429
[cli][build] Relax the git id plugin (#6094)
* [cli][build] Relax the git id plugin

* Guard against null on missing build properties
2020-04-30 08:49:07 -04:00
William Cheng
46216cd796 Merge remote-tracking branch 'origin/master' into 5.0.x 2020-04-28 20:35:28 +08:00
Justin Black
8e8471c1fd
[CORE] Fixes composed schema discriminator map (#4906)
* Adds addComposedMappedModels and testComposedSchemaOneOfDiscriminatorMap

* Requires that discriminators be required properties

* Strengthens discriminaotr validation, adds better error messages, adds schema oneof samples

* Adds oneOf and anyOf invalidDiscriminator tests

* Updates incorrect addOneOfInterfaceModel invocation

* Runs ensure-up-to-date

* Adds updates from Sebastien Rosset

* Removes newlines

* Uses df.isString

* Fixes tests be correctly setting df.isString

* Updates discriminatorExplicitMappingVerbose description per PR feedback

* Adds description of how mappedModels is populated

* Adds the suggestion exception raising when a MappedModel mappingName is null

* Actually resolves merge conflicts

* Switches two methods to package private because they are needed for testing

* Allow nulls in MappedModel.getMappingName

* Updates CLI flag name to legacyDiscriminatorBehavior, default=true

Co-authored-by: William Cheng <wing328hk@gmail.com>
2020-04-25 00:01:02 +08:00
William Cheng
8fa76843d0 resolve merge conflicts in the core generator files 2020-04-17 16:10:10 +08:00
William Cheng
faf6f197be Merge remote-tracking branch 'origin/master' into 5.0-sync-master 2020-04-17 15:51:00 +08:00
Jim Schubert
2957dd4d45
[cli] Add --global-property for -D replacement (#5687)
-D option has been deprecated as it was previously used to:

* Pass "system properties"
* Pass additional properties

This was confusing because we already have --additional-properties and
because Java System Properties are passed as -D before program
arguments.

Confusion around the -D option had existed for some time, but when we
introduced the thread-safe GlobalSettings to avoid overwriting Java
System Properties, we created a hard break from Java System Properties
in the generator. This also disconnected the previous "system
properties" from accepting additional properties.

Once these newly deprecated methods are removed, we will have a clear
separation of concerns between:

* Java System Properties
* Global generator properties (used as workflow context)
* Additional properties (used as generator options)

This commit marks multiple places for cleanup in 5.0. These will be
breaking changes, and lower effort to break in 5.0 with deprecation
warnings now rather than adding sibling properties throughout the code
and potentially introducing logic errors.
2020-04-02 11:11:02 +08:00
Jim Schubert
e14e5fccf3
[cli][docker] Better expose version/sha information of builds (#5736)
* [cli] Some CLI improvements…

* Introduce --version
* Introduce --help
* Add --sha to version command for short SHA display
* Output Version and SHA details
* In new --version output, display repo and doc site

Additional cleanup to suppress warnings and code quality.

* [docker] Adds labels for metadata

This adds image labels to store metadata on the online and cli docker
images, using standard labels:

* org.opencontainers.image.created
* org.opencontainers.image.revision
* org.opencontainers.image.title
* org.opencontainers.image.version

These can be inspected via 'docker inspect IMAGE_NAME' and may be useful
in tooling/automation or bug reports submitted by users.

For more details on these labels, see:
https://github.com/opencontainers/image-spec/blob/master/annotations.md

* Include version --full for equiv to --version
2020-04-02 11:05:25 +08:00
William Cheng
a0ca253218
update samples (#5722) 2020-03-27 13:55:55 +08:00
William Cheng
c224cf484b
4.3.0 release (#5721) 2020-03-27 11:34:43 +08:00
Jim Schubert
f6f5c9b8b9
[cli][gradle] Validate now uses parseOptions w/setResolve (#5471)
* [cli] Validate now uses parseOptions w/setResolve

The validate command now uses ParseOptions#setResolve(true) to match how
we parse in CodegenConfigurator and online's Generate. Without this
option, the OpenAPI 3 parser skips the "resolve" block, which made lead
to validations in the command not matching validations during
generation.

* [gradle] Validate now uses parseOptions w/setResolve

The Graldle validate command now uses ParseOptions#setResolve(true) to match how
we parse in CodegenConfigurator and online's Generate. Without this
option, the OpenAPI 3 parser skips the "resolve" block, which made lead
to validations in the command not matching validations during
generation.
2020-03-01 17:26:13 +08:00
Jim Schubert
36b1a61b70
[all] Move feature set setter (#5460)
When I originally implemented the feature set code, I added the
getter/setter on DefaultCodegen and CodegenConfig as well as on
GeneratorMetadata. GeneratorMetadata also includes the library variation
features. When I went to add library-specific features, I realized the
discrepancy.

This removes the public setter from DefaultCodegen/CodegenConfig, and
adds a protected modifyFeatureSet which accepts a lambda and hides the
builder logic away in the method.

This will be a breaking change for anyone who's created a custom
generator in 4.2.3, so the impact is very limited.
2020-02-28 14:45:06 +08:00
Jim Schubert
db47b95fc9
[cli][core] Add support for dry-run and display (#5332)
* 👕🎨 Minor refactor DefaultGenerator

This cleans up some lint warnings and improve general code cleanliness
of DefaultGenerator.

Specifically:

* logger strings are now using the built-in log formatter rather than
  constructing new strings regardless of log level.
* Diamond operators are used where possible
* Some long-unused commented code has been removed
* Lambdas are used where possible
* Redundant operations are merged (HashMap constructor used rather than
  subsequent putAll on a collection, for example)

* [cli][core] Add support for dry-run and display

CLI now supports `--dry-run`, which will output a file change status
similar to git status --porcelain.

The user may also specify `--verbose` for a one-liner below each file
explaining why the change operation might take place.
2020-02-18 20:36:17 -05:00
Henré Botha
930afadc87
Fix docs for --enable-post-process-file (#5299) 2020-02-13 15:07:07 +08:00
Henré Botha
324fe70ab3
Edit documentation for -c (#5288) 2020-02-13 15:05:52 +08:00
Jim Schubert
aa7cd4f5af
[docs] Upgrade to Docusaurus 2 (#5271)
* [docs] Upgrade to Docusaurus 2

Due to formatting issues with Docusaurus 1 and code blocks, and upcoming
4.3 and 5.0 releases, this is an upgrade to have more control over
formatting and other functionality.

This adds behavior to have light/dark themes.

As a consequence of upgrading, columnar format on ul element in
generated generator docs had to be moved to css. This will not impact
users viewing the markdown on GitHub because it didn't display the
column format.

This upgrade should improve syntax highlighting and performance.

* Fix user page layout, choose theme with clear diff colors
2020-02-11 21:22:58 +08:00
Jim Schubert
8d6286dfae
[core] Normalizing vendor extension naming (#5192)
* [core] Normalizing vendor extension naming

According to [OAS 2.0][1] and [OAS 3.0][2] Specifications:

> Allows extensions to the OpenAPI Schema. The field name MUST begin with x-,
>  for example, x-internal-id. The value can be null, a primitive, an array or an object.
>  Can have any valid JSON format value.

This commit attempts to define a [clear identifier design format][3] of
maintaining lower-kebab casing and following the x- prefix defined by
OAI Specification.

Following a convention that matches that used by others (see [autorest][4]), we will remove
any confusion about naming strategies for template authors and
customizers. Following the lower-kebab convention will allow us to
convert from camelCase and missing prefixes to the desired format. For
example, these conversions are simple to make for template consistency:

* customValue => x-custom-value
* x-customValue => x-custom-value
* x-custom-value => x-custom-value

This convention also allows us to define a single standard for use
across all generators. This means no occurrence of x-operationId in one
generator and x-operation-id in another.

[1]: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#patterned-objects
[2]: https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.2.md#specificationExtensions
[3]: https://tools.ietf.org/html/draft-wilde-registries-01#section-3.4
[4]: https://github.com/Azure/autorest/tree/master/docs/extensions

* Incorporate feedback to avoid race/blocking in OnceLogger

* Remove unnecessary additional log config

* Add tests,comments for OnceLogger

* Test caffeine cache with FakeTicker
2020-02-06 22:52:58 +08:00
William Cheng
100b01844a Merge remote-tracking branch 'origin/master' into 5.0.x 2020-02-04 15:31:10 +08:00
Jim Schubert
97ff9b4be7
[cli] Optional colorized outputs (#5193) 2020-02-02 17:36:53 -05:00
Jim Schubert
a496c2011f
[docs] Initial FeatureSet generator documentation (#5188)
* Flatten feature sets for display

* [cli] Add feature set output to config-help

This includes markdown and plain text outputs for config-help.

* [docs] FeatureSet on generator docs
2020-02-01 20:51:55 -05:00
Jim Schubert
22c6c0ca68
[core] Extracting recommendations to validation framework (#4979)
* [core] Extracting recommendations to validation framework

This is work to extract recommendation logic out of the CLI validate command
into a shared evaluation instance which can be used elsewhere (such as Gradle,
or the Online tool).

For now, these validations are in addition to those provided by swagger-parser and
are only the following recommendations:

* Apache/Nginx warning that header values with underscore are dropped by default
* Unused models/schemas
* Use of properties with oneOf, which is ambiguous in OpenAPI Specification

I've allowed for disabling recommendations via System properties, since this is
something that has been requested a few times by users. System properties in this
commit include:

* openapi.generator.rule.recommendations=false
  - Allows for disabling recommendations completely. This wouldn't include all warnings
    and errors, only those we deem to be suggestions
* openapi.generator.rule.apache-nginx-underscore=false
  - Allows for disabling the Apache/Nginx warning when header names have underscore
  - This is a legacy CGI configuration, and doesn't affect all web servers
* openapi.generator.rule.oneof-properties-ambiguity=false
  - We support this functionality, but the specification may not intend for it
  - This is more to reduce noise
* openapi.generator.rule.unused-schemas=false
  - We will warn when a schema is not referenced outside of Components, which
    users have requested to be able to turn off
* openapi.generator.rule.anti-patterns.uri-unexpected-body=false

* Move recommendation/validations to oas package and add javadoc comments

* Refactor and test recommendation validations

* Refactor validation function signatures to return explicit state rather than boolean

* Add operation recommendation for GET/HEAD w/body
2020-01-31 12:19:16 -05:00
William Cheng
03a59046f4 Merge remote-tracking branch 'origin/4.3.x' 2020-01-31 19:00:28 +08:00
William Cheng
26ace1337d
Prepare 4.2.3 release (#5172)
* release 4.2.3

* update readme
2020-01-31 16:24:06 +08:00
Jim Schubert
c0f7b47292
[ci][test] Modifications to static analysis contribution via Tomasbjerre (#5116)
* Spotbugs, PMD and Checkstyle #33

* Reducing Spotbugs effort to min #33

 * Also using project.parent.basedir and avoiding relative paths in pom files.
 * Filtering out samples.

* Move PMD/Spotbugs to static-analysis profile

This moves the static-analysis checks to a standalone profile. Core
contributors may run static analysis with:

```
mvn -Pstatic-analysis install
```

The analysis is separated from default functionality to reduce impact to
community contributions. SpotBugs/PMD may add a non-trivial amount of
time to builds on some machines.

Co-authored-by: Tomas Bjerre <tomas.bjerre85@gmail.com>
2020-01-26 12:45:41 -05:00
Jim Schubert
c9ec084418
🐛 Fixing some issues with threading and NPE (#5107)
* 🐛 Fixing some issues with threading and NPE

After running Sonar on the master branch, some major analysis
opportunities were displayed.

This fixes the use of SimpleDateFormat stored as static fields.
SimpleDateFormat is not thread-safe, and may retain data across threads.
While there's no indicator that this has caused any issues (these are
mostly used for example code), we should follow these best practices.

This also fixes a handful of NPE and other minor issues such as
comparing Boolean.TRUE to strings and no wrapping some closeables in
try-with-resources.

* [cli] Unit test GenerateBatch custom deserialization helper

* Quiet batch mode in sonar.yml

* Suppress unnecessary warnings (ThreadLocals in static fields)
2020-01-25 18:28:16 -05:00
Jim Schubert
ac528aaf07
[test] Removes jmockit in favor of mockito (#5063)
* [test] Removes jmockit in favor of mockito

We use mockito in many tests. This removes jmockit which is run as a
javaagent in favor of Mockito which is not.

This work is in preparation for applying some static analysis tools,
while evaluating others such as Jacoco. I'm also look at ways to improve
build times while also decreasing "ramp up time" for contributions from
the community. Reducing the number of mock frameworks and dependencies
is a step toward that goal.

* Rename method in new.sh

* [cli] Mock the generate task
2020-01-22 18:04:00 -05:00
William Cheng
82e2448bf2 Merge remote-tracking branch 'origin/4.3.x' into 5.0.x 2020-01-20 15:46:13 +08:00
William Cheng
04af5e0445 Merge remote-tracking branch 'origin/master' into 4.3.x 2020-01-20 15:04:34 +08:00
Jim Schubert
644f720db8
[docs] Sorted doc outputs and clean up duplicated CliOptions (#5046)
* [config-help] Sort all outputs

* Remove old/stale generator docs (these have been renamed previously)

* Sort config doc outputs, making it easier to find relevant info

* Fixes cliOptions duplicates

Erlang client/proper improperly set the version option as packageName
(causing duplicate for packageName).

The java and swift option removals are because the options are added in
parent classes, resulting in duplication of the options.

* Add --github-nested-index for generating docs/generators/README.md

* 📝 Regenerate generator docs
2020-01-19 20:19:11 -05:00
Jim Schubert
ea55968737
Evaluating https issues in CircleCI (on top of CiscoM31-maven-h… (#5034)
* replace http with https. Replace http://central.maven.org with https://repo1.maven.org

* replace http://www.apache.org/licenses/LICENSE-2.0 with https://www.apache.org/licenses/LICENSE-2.0

* Force HTTPS for mavenCentral gradle DSL

Co-authored-by: Sebastien Rosset <serosset@cisco.com>
2020-01-18 18:53:05 -05:00