| Re: [jsonb-dev] [External] : @JsonbRequired for JSONB and JSON Schema |
_______________________________________________Before accepting criticism, I would want to make sure that you folks fully understand the use case. We are making a backwards incompatible change by not requiring a pretense of @JsonbCreator parameters in your JSON. How will you deal with the use case when you actually want these parameters to be present? We must deal with it because otherwise there is a space for uncovered use cases.
-- Dmitry
From: jsonb-dev <jsonb-dev-bounces@xxxxxxxxxxx> On Behalf Of Romain Manni-Bucau
Sent: 16 November 2021 16:17
To: jsonb developer discussions <jsonb-dev@xxxxxxxxxxx>
Cc: Jakarta specification discussions <jakarta.ee-spec@xxxxxxxxxxx>
Subject: Re: [jsonb-dev] [External] : @JsonbRequired for JSONB and JSON Schema
Ok so since there is an agreement on relaxing the spec, let's make it in main branch and since there is a disagreement on @JsonbRequired let's just drop it and not brute force it to be there without being able to justify it nor integrate it in jakarta ecosystem (yet).
To answer David's question, right know I'm using this kind of validator on JsonObject level (https://github.com/apache/johnzon/blob/master/johnzon-jsonschema/src/test/java/org/apache/johnzon/jsonschema/JsonSchemaValidatorTest.java#L53) and then bind the JsonObject to a POJO/record (both Johnzon and Yasson have this API now, no more sure the spec got it properly too but it avoids another serialization/parsing round trip so perfs are good).
A better and more scalable approach for stream is to do it at parser level (sorry this one is not public) but long story short it is decorating the parser to add validation to each events. On a spec level I would expect something like:
final var validator = new Validator();
// or
final var validator = Json.createSchemaFactory(Map.of()).createShemaValidator(myJsonObjectJsonSchema);
// then use it
final var parserFactory = Json.createParserFactory(Map.of(JsonParser.VALIDATOR, validator));
// indeed by "transitivity"
final var readerFactory = Json.createReaderFactory(Map.of(JsonParser.VALIDATOR, validator));
A nicer option would be to enable to pass a map to the parser itself to not have to share it globally and have multiple instances of factories (current JSON-P limitation, it is the same for prettification for ex).
Currently it is more done this way since this API is not there:
final var myValidatedParser = new ValidatingParser(parserFactory.createParser(...), myJsonObjectJsonSchema);
But this is an implementation detail, the key is that in terms of design there is no real choice to do the validation at JSON-P level - and more precisely parser to enable more advanced validation on huge payloads).
If you want to talk about validation (like ensuring an attribute has a value in the payload for ex) we need to ensure:
1. It covers the common needs and not just the most trivial to check,
2. It is integrated to the ecosystem (failure handler, compatibility with ajv for ex, ..... JAXB is a good example there),
3. It is done properly at the right level, ie JSON-P and JSON-B just wires the needed config (don't forget JSON-B is written to accept both JsonReader or JsonParser usages by constructions and to be a mapper API and not an implementation which would defeat the spec target by design - enables to not be vendor locked in, otherwise no point using a spec ;))
4. We can use it as a base for other specs (like MP openapi for ex instead of duplicating all the API, in particular if MP joins jakarta at some point)
I guess it is a big topic we can try to address for next big release - or just do an extension/appendix to propose it in the spec - but it is likely too much for next immediate one and if we do it in a hurry it would be wrong again and deserve us so let's just drop it for now, no?
Le mar. 16 nov. 2021 à 15:58, Jean-Louis Monteiro <jlmonteiro@xxxxxxxxxxxxx> a écrit :
--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro
http://www.tomitribe.com
On Tue, Nov 16, 2021 at 3:54 PM Erik Mattheis via jsonb-dev <jsonb-dev@xxxxxxxxxxx> wrote:
The core issue is the mismatch between JSON and Java as it pertains to detecting the presence or absence of a property. Using traditional JavaBean mechanics, it is possible to differentiate between the absence of a property in the JSON payload (setter is never called) versus a property with a null value (setter is called with a null argument). For creator methods/constructors, there is no direct way to distinguish between an absent value and a null value and the spec (unfortunately) mandates that an exception is thrown for missing fields.
Agreed.
Relaxing the spec to permit the deserialization of JSON documents with absent fields using creator methods/constructors and addressing backwards compatibility is the sole issue at hand. Unfortunately, the current spec behavior blurs the line between this issue and validation because any code that relies on creators throwing an exception when a field is absent is implicitly relying on JSON-B to perform validation.
Agreed
Personally, I’m in favor of simply relaxing the spec to permit the deserialization of absent fields as null values in creators and making it clear that it is not possible to differentiate between null and absent properties. I believe that is what users of creators expect, and anyone who needs to differentiate the two cases can use setters.
Agreed
I agree that @JsonbRequired isn’t a great idea here. It makes the expectation of validation explicit where it was previously implied. It would be interesting to survey usage of JSON-B to see if anyone really wants this. As Romain points out, it’s not a great solution for backwards-compatibility because it requires changing code anyway. A better solution (in my opinion) would be to allow programmer to specify the value to use when a field is absent (e.g. @JsonbDefault(“value”) or something similar). This would match the specified behavior for setters (setter must not be called so default value is preserved) and keeps JSON-B focused on the mechanics of deserialization instead of straying into the realm of validation.
Also agreed but don't think we need any solution for now other than relax the specification that mandates an exception.
—
Erik Mattheis
On Nov 16, 2021, at 8:06 AM, David Kral <david.k.kral@xxxxxxxxxx> wrote:
> If you want to restore previous behavior for existing app you must have a *system property* - or env var
Sure, feel free to have it on impl level. That’s exactly something what belongs there.
> This is a wrong solution by construction and it does not reach its functional goal.
Its goal is not to create any extended validations, so I think it meets its goal just fine. And because of that I do not believe anything like Bean Validation integration (or even schema) should be included. This is there just able to turn “the old behaviour” on the specific parameter/creator.
> Most (all actually) users needing validations need more than that so can't use that and use something else ….
I don’t think so. This is more than enough. If anyone wants to do the value validation, sure, feel free to add it into the creator itself. I do not see how this change of ours would make creator “unusable” in the most cases. As mentioned, several times, it is there to provide backwards compatible functionality, without the need of defining it globally.
> JSON-P needs a validation (at parser level likely)
I have to say, I am not sure, how this would improve user experience or even how it solves the problem we solved with the @JsonbRequired. Do you have any example which would illustrate the solution and how user would use it with JSONB?
David
Od: jsonb-dev <jsonb-dev-bounces@xxxxxxxxxxx> za uživatele Romain Manni-Bucau
Odesláno: úterý 16. listopadu 2021 13:34
Komu: jsonb developer discussions <jsonb-dev@xxxxxxxxxxx>
Kopie: Jakarta specification discussions <jakarta.ee-spec@xxxxxxxxxxx>
Předmět: Re: [jsonb-dev] [External] : @JsonbRequired for JSONB and JSON Schema
Le mar. 16 nov. 2021 à 12:38, David Kral <david.k.kral@xxxxxxxxxx> a écrit :
Hi Jean-Louis,
I don’t think we are getting here into the area of bean validation. This is not about validation of the content in any way.
The PR introduced backwards incompatible change to the creators, since the parameters are no longer required by default, some other mechanism for specifying required parameters needed to be introduced. @JsonbRequired annotation has been added mainly to provide users ability to specify which parameters are expected to be present in the JSON upon object creation. It does not care if the value has been null or value. It is simple requirement of some property to be present. If the required property is not present -> fail.
This is a wrong solution by construction and it does not reach its functional goal. If you want to restore previous behavior for existing app you must have a *system property* - or env var - cause otherwise it requires a new compilation and code change which means you don't need this annotation in such cases anyway.
I do not agree it is messy. The way I see it, is the same behaviour as we used to have, and no extended bean validation was required either. Therefore, I do not think doing any bean validation integration or even schema creation make sense.
There it is also part of the story only.
Most (all actually) users needing validations need more than that so can't use that and use something else like a jsonschema based validator or a custom one - often in interceptors or filters - for the most common cases I saw.
So having a built-in validation nobody uses - or intend to use this way - is likely wrong and in terms of consistency with the ecosystem will make the user life harder.
JSON-P needs a validation (at parser level likely), so once it is there JSON-B will just wire it providing a JsonProvider in JsonbConfig or something like that and voilà.
This is the level JSON-B must integrate with the ecosystem.
Currently the validation is super rude - no way to recover - and so incomplete that it is disabled in most applications.
Also, if we can split in 2 the topics:
1. relax the instantiation - everybody agreed on that and consider current spec as a blocker
2. required handling or not: there, there is no agreement so maybe just postpone it instead of forcing the merge?
David
Od: jsonb-dev <jsonb-dev-bounces@xxxxxxxxxxx> za uživatele Jean-Louis Monteiro via jsonb-dev
Odesláno: úterý 16. listopadu 2021 12:12
Komu: jsonb-dev@xxxxxxxxxxx; jsonp-dev@xxxxxxxxxxx
Kopie: Jean-Louis Monteiro <jlmonteiro@xxxxxxxxxxxxx>; Jakarta specification discussions <jakarta.ee-spec@xxxxxxxxxxx>
Předmět: [External] : [jsonb-dev] @JsonbRequired for JSONB and JSON Schema
Hi all,
I’d like to follow up on some discussions around the @JsonbRequired introduced.
On the feature itself, I clearly see some benefits to have validation happening before the bean is constructed and fields populated with the JSON payload. Good feature.
On the other hand, I don’t think @JsonbRequired is a good way to achieve it.
Basically I see 2 options for validation
- payload validation
- bean validation
@JsonbRequired in the bean to do payload validation does not seem accurate to me. It will set us for failure in the future.
If we want to address bean validation, we should consider having some integration of Bean Validation using Bean Validation annotations. Of course, this means validation will happen after the bean is constructed, but it also is way richer than “is the field required or not”. Users will be able to reuse constraints and implement business logic validation.
I agree that validating the payload before the object gets constructed is very interesting. But then, we should use JSON Schema for it. I am ok to create JSON Schema equivalent annotations, at least for the stable subset and the most interesting rules. From there, we can generate the JSON Schema, and feed the JSON Parser with it so the parser can effectively validate the payload before with a valid JSON Schema (same way an XML parser would get the XSD to validate the payload in JAXB).
Having @JsonbRequired seems to be in the middle and a hack for a very simple use case. I would not introduce it now and give us more time to discuss and think about the payload validation a bit more.
The hack has been introduced to solve an issue with the design of @JsonbCreator. See https://github.com/eclipse-ee4j/jsonb-api/issues/121
Again the design for @JsonbCreator and the need for the feature is awesome for immutable objects. The way to solve it with @JsonbRequired is messy.
Moreover, if we look at MicroProfile and OpenAPI, we are also half way because OpenAPI has @Schema, for instance
@Schema(type = SchemaType.OBJECT, implementation = InventoryList.class)
or
@Schema(required = true)
private final String hostname;
From there, we generate an openapi document with all constraints (not a JSON Schema but pretty close).
Let’s say we need business validation or cross fields validations, we’ll use Bean Validation.
Let’s say we want OpenAPI documentation, we’ll use OpenAPI @Schema
And now let’s say we want JSON Payload validations, we’ll use @JsonbRequired?
Can we maybe consider the schema validation as a whole?
We could use the same set of annotations to generate the OpenAPI documentation and to generate the JSON Schema for the JSON Parser.
Thoughts?
I am happy to create an issue to revert @JsonbRequired if there is a consensus to wait.
--
Jean-Louis Monteiro
http://twitter.com/jlouismonteiro
http://www.tomitribe.com_______________________________________________
jsonb-dev mailing list
jsonb-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jsonb-dev_______________________________________________
jsonb-dev mailing list
jsonb-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jsonb-dev
_______________________________________________
jsonb-dev mailing list
jsonb-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jsonb-dev_______________________________________________
jsonb-dev mailing list
jsonb-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jsonb-dev
jsonb-dev mailing list
jsonb-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jsonb-dev