Changelog History
Page 108
-
v0.15.2 Changes
(RB=44099)
Internal changes to replace Stack with ArrayList or ArrayDeque.
-
v0.15.1 Changes
- Main API change is removal/decoupling of validation from DataSchema. DataSchema no longer has validate method. The replacement is ValidateDataAgainstSchema.validate(...). ⬇️ Reduce memory allocation for DataElement for each object visited. Will reuse same DataElement for each member of a container. As part of this change, it is no longer possible to get a 🏗 standard Iterator from a builder. The alternative is to use the traverse method that takes a callback for each object iterated. ➕ Add support for different pre-order and post-order traversal to ObjectIterator. This allows ObjectIterator to be used for data to schema validation. This unification allows single pass data to schema validation as well as calling Validator after 🛠 fixup and schema validation. ✨ Enhance DataSchemaAnnotationValidator to not throw exception on construction. Allow validator to be used if only some validators are constructed. Use common Message classes for emitting 🎉 initialization messages. 🔨 Refactor code to allow both iterative and recursive validation. ➕ Add more test cases.
- ➕ Add support to taking DataElement as starting point for iterating through Data objects and for validation. This has been requested by Identity superblock where the patch is applied to position (using position as starting point), but the root object is a profile. The validation should start 🔌 where the patch is applied, but the validator plugin wants access to the entire entity, i.e. the profile entity. ➕ Add tests and bug fix unnecessary additional calls to validators from ValidateDataAgainstSchema when typerefs are in use. The 🐛 bug was that the downstream validator will be called once per 🛠 typeref in the typeref chain. The correct and fixed behavior is that the downstream validator will be called once per data object (not once per schema typeref'ed). 0.15 ➕ Add pluggable validator to Data Schemas
- Change behavior of ObjectIterator to include returning the input value. 👀 2. See com.linkedin.data.validator package and TestValidator class for how to use validators.
- This is still a prototype feature. Output Avro-compliant equivalent JSON from Pegasus schema ➕ Add translation between Pegasus DataMap and Avro GenericRecord 🔨 1. Also include refactoring of DataSchema to JSON encoding to move Avro specific code out of cow module into cow-avro module. ⚡️ Rest.li support for full and partial updates Full update (overwrite) is transported as an HTTP PUT to the entity URI, 🛰 with a payload containing the JSON serialized RecordTemplate of the entity schema. Partial update (patch) is transported as an HTTP POST to the entity URI, 🛰 with a payload containing a JSON serialized PatchRequest The internal structure of a PatchRequest is documented here: ⚡️ https://iwww.corp.linkedin.com/wiki/cf/display/ENGS/Partial+Update PatchRequests can be generated on the client side by "diff'ing" two RecordTemplate objects using PatchGenerator in com.linkedin.restli.client.utility. Patch Generation relies on the facilities from the data-transform pegasus 📦 component, in the com.linkedin.data.transform package. PatchRequests can be applied on the server side by providing a pre-image RecordTemplate object and a PatchRequest to PatchApplier in com.linkedin.restli.server.util. Patch application uses the DataMapProcessor from the pegasus data-transform component. ⚡️ Full and Partial update are provided as overloaded update() methods in CollectionResource/AssociationResource on the server-side and as overloaded ⚡️ buildUpdate() methods in EntityRequestBuilder on the client-side. ⚡️ PARTIAL_UPDATE is defined as a new ResourceMethod, and listed as appropriate 👍 in the IDL "supports" clause of the resource. 👌 Support for deep (nested) projections has been implemented: Server-side, the rest.li framework understands both the old "field1,field2,field3" 💅 syntax and the new PAL-style "field1,field2:(nestedfield)" syntax. Projections are applied automatically by the framework, using pegasus data-transform. ResourceContext provides access to the projections as either a Set or a MaskTree. Client-side, the generated RecordTemplate classes have been modified to provide fields as a nested class accessed through the .fields() static method. Each field can be accessed as a Path object through a fieldName() method, which provides full static typing. Fields are provided as methods rather than member variables to avoid initialization cycles when a RecordTemplate contains a field of the same type. Deep projection support is currently disabled on the client side, to avoid ordering 🚀 issues with deployment. Once all services have been deployed to production with the 🆕 new pegasus version, we will enable deep projections on the client side. 👍 More background on projection support is available here: https://iwww.corp.linkedin.com/wiki/cf/display/ENGS/Projections Compatibility: ⚡️ WIRE: This change is wire INCOMPATIBLE for existing users of update. There are ⚡️ no known uses of update in production. ⚡️ CLIENT: This change is library INCOMPATIBLE for users of update, projection fields, 🏗 or code that relies on the types of framework ...Builder classes (a type parameter has ⬆️ been removed). Client code will need to be upgraded. ⚡️ SERVER: This change is library INCOMPATIBLE for users of update. Server code will ⬆️ need to be upgraded to use PatchRequests and PatchApplier. 🔨 Validation code refactoring. 🚚 1. Move validation code out of DataSchema classes, enable single pass validation of schema and calling validator plugins. 📦 2. Move schema validation code into validation package. 📦 3. Move validator plugins into validator package.
- Provide field specific diagnostic data from schema validator. 📇 Rename cow and cow-avro module to data and data-avro module. 🔨 Refactor Cow classes and provide implementation that have the checker functionality but without copy-on-write, known as CheckedMap and CheckedList classes.
- ➕ Add support for "validate" annotation on fields. Prefix DataElement accessor methods with "get".
- ➕ Add support for filtering calls to a Validator based on what has been set as specified by the patch operations map. This functionality is implemented by the PatchFilterValidator class. Refactored ValidationMessage* classes into Message* classes so that patch operations can re-use these classes for generating patch messages.
-
v0.15 Changes
➕ Add pluggable validator to Data Schemas (RB=41693)
- Change behavior of ObjectIterator to include returning the input value.
👀 2. See com.linkedin.data.validator package and TestValidator class for how to use validators.
- This is still a prototype feature.
Output Avro-compliant equivalent JSON from Pegasus schema (RB=41878)
➕ Add translation between Pegasus DataMap and Avro GenericRecord (RB=42130)
🔨 1. Also include refactoring of DataSchema to JSON encoding to move Avro specific code out of cow module into cow-avro module.
⚡️ Rest.li support for full and partial updates Full update (overwrite) is transported as an HTTP PUT to the entity URI, 🛰 with a payload containing the JSON serialized RecordTemplate of the entity schema.
Partial update (patch) is transported as an HTTP POST to the entity URI, 🛰 with a payload containing a JSON serialized PatchRequest
The internal structure of a PatchRequest is documented here: ⚡️ https://iwww.corp.linkedin.com/wiki/cf/display/ENGS/Partial+Update
PatchRequests can be generated on the client side by "diff'ing" two RecordTemplate objects using PatchGenerator in com.linkedin.restli.client.utility. Patch Generation relies on the facilities from the data-transform pegasus 📦 component, in the com.linkedin.data.transform package.
PatchRequests can be applied on the server side by providing a pre-image RecordTemplate object and a PatchRequest to PatchApplier in com.linkedin.restli.server.util. Patch application uses the DataMapProcessor from the pegasus data-transform component.
⚡️ Full and Partial update are provided as overloaded update() methods in CollectionResource/AssociationResource on the server-side and as overloaded ⚡️ buildUpdate() methods in EntityRequestBuilder on the client-side.
⚡️ PARTIAL_UPDATE is defined as a new ResourceMethod, and listed as appropriate 👍 in the IDL "supports" clause of the resource.
👌 Support for deep (nested) projections has been implemented: Server-side, the rest.li framework understands both the old "field1,field2,field3" 💅 syntax and the new PAL-style "field1,field2:(nestedfield)" syntax. Projections are applied automatically by the framework, using pegasus data-transform. ResourceContext provides access to the projections as either a Set or a MaskTree.
Client-side, the generated RecordTemplate classes have been modified to provide fields as a nested class accessed through the .fields() static method. Each field can be accessed as a Path object through a fieldName() method, which provides full static typing. Fields are provided as methods rather than member variables to avoid initialization cycles when a RecordTemplate contains a field of the same type.
Deep projection support is currently disabled on the client side, to avoid ordering 🚀 issues with deployment. Once all services have been deployed to production with the 🆕 new pegasus version, we will enable deep projections on the client side.
👍 More background on projection support is available here: https://iwww.corp.linkedin.com/wiki/cf/display/ENGS/Projections
Compatibility: ⚡️ WIRE: This change is wire INCOMPATIBLE for existing users of update. There are ⚡️ no known uses of update in production.
⚡️ CLIENT: This change is library INCOMPATIBLE for users of update, projection fields, 🏗 or code that relies on the types of framework ...Builder classes (a type parameter has ⬆️ been removed). Client code will need to be upgraded.
⚡️ SERVER: This change is library INCOMPATIBLE for users of update. Server code will ⬆️ need to be upgraded to use PatchRequests and PatchApplier.
🔨 Validation code refactoring.
🚚 1. Move validation code out of DataSchema classes, enable single pass validation of schema and calling validator plugins. 📦 2. Move schema validation code into validation package. 📦 3. Move validator plugins into validator package.
- Provide field specific diagnostic data from schema validator.
📇 Rename cow and cow-avro module to data and data-avro module.
🔨 Refactor Cow classes and provide implementation that have the checker functionality but without copy-on-write, known as CheckedMap and CheckedList classes.
(RB=42905)
➕ Add support for "validate" annotation on fields.
Prefix DataElement accessor methods with "get".
(RB=43002)
➕ Add support for filtering calls to a Validator based on what has been set as specified by the patch operations map. This functionality is implemented by the PatchFilterValidator class.
Refactored ValidationMessage* classes into Message* classes so that patch operations can re-use these classes for generating patch messages.
-
v0.14.7 Changes
- 🛠 Fix a bug in D2 where the PropertyStore information was not correctly persisted to disk, preventing the load balancer from operating correctly if connectivity to ZooKeeper was interrupted.
-
v0.14.6 Changes
(RB=41257)
👍 1. Add support for typeref to enable aliasing primitive types. Typeref works for any type (not just primitive). There is no support yet for binding different Java classes to the typeref'ed types. If a typeref is member of union, then the accessor method names for accessing the member are derived from the typeref name.
Serialization protocol and format does not change. The serialized representation of the member key is always the actual type, i.e. the type reached by following chain of typerefs.
Rest.li @Action and @ActionParam now has optional 'typeref' attribute that specifies the typeref that should be emitted to the IDL instead of the type inferred from the Java type. The provided typeref must be compatible with the Java type.
KNOWN ISSUE - Unions cannot be used as return or parameter for actions.
-
v0.14.5 Changes
(RB=41200)
- Provide human readable locations when Jackson parse errors occurs.
-
v0.14.4 Changes
Data Schema Resolver (RB=41014)
RUM pegasus plugin will not have to unjar before passing data schema jars to pegasus. See seperate review for changes to plugin.
Remove location from in-memory representation of schemas. Location is only used for generating Java bindings and checking for freshness of generated files. It is not needed to in-memory representation. Storing them in in-memory bindings may cause file "leaks" as they refer to jar files and files.
Simply by removing FIELDS_ONLY record type.
-
v0.14.3 Changes
(RB=40353)
Minor changes to comments in rpc-demo-client.
Fix pdpr and avsc references in tools/rest-doc-generator/docgen.py
-
v0.14.2 Changes
- 🚀 Not released.