Skip to main content

transformer.proto

path mgmt/v1alpha1/transformer.proto

package mgmt.v1alpha1


Messages

CreateUserDefinedTransformerRequest

NameTypeDescription
account_idstringThe unique identifier of the account to create the user defined transformer for.
namestringThe name of the user defined transformer.
descriptionstringThe description of the user defined transformer.
sourceTransformerSourceThe source of the user defined transformer.
transformer_configTransformerConfigThe configuration of the user defined transformer.

CreateUserDefinedTransformerResponse

NameTypeDescription
transformerUserDefinedTransformer

DeleteUserDefinedTransformerRequest

NameTypeDescription
transformer_idstringThe unique identifier of the user defined transformer to delete.

DeleteUserDefinedTransformerResponse

NameTypeDescription

GenerateBool

NameTypeDescription

GenerateBusinessName

NameTypeDescription

GenerateCardNumber

NameTypeDescription
valid_luhnoptional boolWhether or not to validate the card number using the Luhn algorithm.

GenerateCategorical

NameTypeDescription
categoriesoptional stringA comma separated list of categories that will be used to generate a random value from.

GenerateCity

NameTypeDescription

GenerateCountry

NameTypeDescription
generate_full_nameoptional boolAn option to return the full country name of the randomly selected country or return the default of a 2-letter country code.

GenerateDefault

NameTypeDescription

GenerateE164PhoneNumber

NameTypeDescription
minoptional int64The minimum length of the phone number to generate.
maxoptional int64The maximum length of the phone number to generate.

GenerateEmail

NameTypeDescription
email_typeoptional GenerateEmailTypeOptionally specify the type of email to generate. The types specified determine the contents on the left side of the @.

GenerateFirstName

NameTypeDescription

GenerateFloat64

NameTypeDescription
randomize_signoptional boolWhether or not to randomize the sign of the float64.
minoptional doubleThe minimum value to randomize the float64 to.
maxoptional doubleThe maximum value to randomize the float64 to.
precisionoptional int64The precision of the float64 to generate.

GenerateFullAddress

NameTypeDescription

GenerateFullName

NameTypeDescription

GenerateGender

NameTypeDescription
abbreviateoptional boolWhether or not to abbreviate the gender.

GenerateInt64

NameTypeDescription
randomize_signoptional boolWhether or not to randomize the sign of the int64.
minoptional int64The minimum value to randomize the int64 to.
maxoptional int64The maximum value to randomize the int64 to.

GenerateInt64PhoneNumber

NameTypeDescription

GenerateIpAddress

NameTypeDescription
ip_typeoptional GenerateIpAddressTypeOptionally specify the type of ip address to generate.

GenerateJavascript

NameTypeDescription
codestringThe user provided javascript code that will be executed in the transformer

GenerateLastName

NameTypeDescription

GenerateSSN

NameTypeDescription

GenerateSha256Hash

NameTypeDescription

GenerateState

NameTypeDescription
generate_full_nameoptional boolAn option to return the full state name of the randomly selected state or return the default of a 2-letter state code.

GenerateStreetAddress

NameTypeDescription

GenerateString

NameTypeDescription
minoptional int64The minimum length of the string to generate.
maxoptional int64The maximum length of the string to generate.

GenerateStringPhoneNumber

NameTypeDescription
minoptional int64The minimum length of the phone number to generate.
maxoptional int64The maximum length of the phone number to generate.

GenerateUnixTimestamp

NameTypeDescription

GenerateUsername

NameTypeDescription

GenerateUtcTimestamp

NameTypeDescription

GenerateUuid

NameTypeDescription
include_hyphensoptional boolWhether or not to include hyphens in the uuid. Defaults to true. If false, the uuid will be returned without hyphens.

GenerateZipcode

NameTypeDescription

GetSystemTransformerBySourceRequest

NameTypeDescription
sourceTransformerSourceThe source of the transformer to retrieve.

GetSystemTransformerBySourceResponse

NameTypeDescription
transformerSystemTransformer

GetSystemTransformersRequest

NameTypeDescription

GetSystemTransformersResponse

NameTypeDescription
transformersrepeated SystemTransformer

GetTransformPiiEntitiesRequest

NameTypeDescription
account_idstringThe unique identifier of the account to return entities for.

GetTransformPiiEntitiesResponse

NameTypeDescription
entitiesrepeated stringThe list of built-in entities that are enabled for the provided account id.

GetUserDefinedTransformerByIdRequest

NameTypeDescription
transformer_idstringThe unique identifier of the user defined transformer to get.

GetUserDefinedTransformerByIdResponse

NameTypeDescription
transformerUserDefinedTransformer

GetUserDefinedTransformersRequest

NameTypeDescription
account_idstringThe unique identifier of the account to get the user defined transformers for.

GetUserDefinedTransformersResponse

NameTypeDescription
transformersrepeated UserDefinedTransformer

IsTransformerNameAvailableRequest

NameTypeDescription
account_idstringThe unique identifier of the account to check for the availability of the transformer name.
transformer_namestringThe name of the transformer to check for availability.

IsTransformerNameAvailableResponse

NameTypeDescription
is_availableboolWhether or not the transformer name is available.

Null

NameTypeDescription

Passthrough

NameTypeDescription

PiiAnonymizer

NameTypeDescription
replacePiiAnonymizer.ReplaceConfigures the anonymizer to replace the identified PII with a specific value
redactPiiAnonymizer.RedactConfigures the anonymizer to completely remove the identified PII
maskPiiAnonymizer.MaskConfigures the anonymizer to mask the identified PII
hashPiiAnonymizer.HashConfigures the anonymizer to hash the identified PII
transformPiiAnonymizer.TransformConfigures the anonymizer to use a Neosync configured transformer

PiiAnonymizer.Hash

NameTypeDescription
algooptional PiiAnonymizer.Hash.HashTypeThe hashing algorithm to use.

PiiAnonymizer.Mask

NameTypeDescription
masking_charoptional stringProvide a character that will be used for the replacement.
chars_to_maskoptional int32How many characters to mask.
from_endoptional boolWhether to mask the PII from start or end

PiiAnonymizer.Redact

NameTypeDescription

PiiAnonymizer.Replace

NameTypeDescription
valueoptional stringThe value to replace. If not provided, a template token of the anonymizer is used (e.g. A PERSON entity is replaced with: <PERSON>)

PiiAnonymizer.Transform

NameTypeDescription
configTransformerConfigThe transformer to use. If not provided, a transformer will automatically be selected (if supported), otherwise it falls back to generating a random hash.

PiiDenyRecognizer

NameTypeDescription
namestringFriendly name of this entity
deny_wordsrepeated stringList of words that will be treated as PII.

SystemTransformer

A system transformer is a transformer that is built into the Neosync platform. It can be used directly in job mappings or used in user-defined transformers to snapshot a specific configuration.

NameTypeDescription
namestringThe name of the transformer.
descriptionstringThe description of the transformer.
data_typeTransformerDataType@deprecated: data_types is deprecated and will be removed in a future release. Use data_types instead.
sourceTransformerSourceThe source of the transformer.
configTransformerConfigThe configuration of the transformer.
data_typesrepeated TransformerDataTypeThe data types supported by the transformer.
supported_job_typesrepeated SupportedJobType

TransformCharacterScramble

NameTypeDescription
user_provided_regexoptional stringA user provided regular expression that will be used to scramble the characters in the input string. Must be a valid Golang regex.

TransformE164PhoneNumber

NameTypeDescription
preserve_lengthoptional boolWhether or not to preserve the original length of the phone number. This causes the transformed phone number to retain the original length.

TransformEmail

NameTypeDescription
preserve_domainoptional boolWhether or not to preserve the original domain, barring what has been specified in the excluded_domains property.
preserve_lengthoptional boolWhether or not to preserve the original length of the email. This causes the transformed email to retain the original length.
excluded_domainsrepeated stringA lsit of email domains that should be excluded. This changes based on the preserve_domain flag. See the docs for more details.
email_typeoptional GenerateEmailTypeOptionally specify the type of email to generate. The types specified determine the contents on the left side of the @.
invalid_email_actionoptional InvalidEmailActionOptionally specify an action in the result of an invalid email. An invalid email is one that does not conform to RFC 5322.

TransformFirstName

NameTypeDescription
preserve_lengthoptional boolWhether or not to preserve the original length of the first name. This causes the transformed first name to retain the original length.

TransformFloat64

NameTypeDescription
randomization_range_minoptional doubleThe minimum value to randomize the float64 to. This will be relative to the original value.
randomization_range_maxoptional doubleThe maximum value to randomize the float64 to. This will be relative to the original value.

TransformFullName

NameTypeDescription
preserve_lengthoptional boolWhether or not to preserve the original length of the full name. This causes the transformed full name to retain the original length.

TransformInt64

NameTypeDescription
randomization_range_minoptional int64The minimum value to randomize the int64 to. This will be relative to the original value.
randomization_range_maxoptional int64The maximum value to randomize the int64 to. This will be relative to the original value.

TransformInt64PhoneNumber

NameTypeDescription
preserve_lengthoptional boolWhether or not to preserve the original length of the phone number. This causes the transformed phone number to retain the original length.

TransformJavascript

NameTypeDescription
codestringThe user provided javascript code that will be executed in the transformer

TransformLastName

NameTypeDescription
preserve_lengthoptional boolWhether or not to preserve the original length of the last name. This causes the transformed last name to retain the original length.

TransformPhoneNumber

NameTypeDescription
preserve_lengthoptional boolWhether or not to preserve the original length of the phone number. This causes the transformed phone number to retain the original length.

TransformPiiText

NeosyncCloud/Enterprise only transformer for anonymizing PII Text

NameTypeDescription
score_thresholdfloatMinimal detection score for determining PII. 0.0-1.0
default_anonymizerPiiAnonymizerThe default anonmyization config used for all instances of detected PII.
deny_recognizersrepeated PiiDenyRecognizerConfigure deny lists where each word is treated as PII.
allowed_entitiesrepeated stringConfigure a list of entities to be used for PII analysis. If not provided or empty, all entities are considiered If this is specified, any ad-hoc, or deny_recognizers entity names must also be provided. To see available builtin entities, call the GetPiiTextEntities() RPC method to see what is available for your account.
allowed_phrasesrepeated stringExact match of PII phrases identified by the analysis engine.
languageoptional stringTwo characters for the desired language in ISO_639-1 format. If not provided, will use a default language if specified on the server. Depending on the server configuration, may attempt to autodetect from input.
entity_anonymizersrepeated TransformPiiText.EntityAnonymizersEntryA map of entity names to anonymizer configurations. The key in this map corresponds to a recognized entity (e.g. PERSON, PHONE_NUMBER) which can be found via the GetPiiTextEntities() RPC method. The value in this map corresponds to the anonymizer configuration to use for that entity. If not provided, the default_anonymizer will be used.

TransformPiiText.EntityAnonymizersEntry

NameTypeDescription
keystring
valuePiiAnonymizer

TransformString

NameTypeDescription
preserve_lengthoptional boolWhether or not to preserve the original length of the string. This causes the transformed string to retain the original length.

TransformUuid

NameTypeDescription

TransformerConfig

The configuration for a transformer. This determines what type of transformation or generation is performed.

NameTypeDescription
generate_email_configGenerateEmail
transform_email_configTransformEmail
generate_bool_configGenerateBool
generate_card_number_configGenerateCardNumber
generate_city_configGenerateCity
generate_e164_phone_number_configGenerateE164PhoneNumber
generate_first_name_configGenerateFirstName
generate_float64_configGenerateFloat64
generate_full_address_configGenerateFullAddress
generate_full_name_configGenerateFullName
generate_gender_configGenerateGender
generate_int64_phone_number_configGenerateInt64PhoneNumber
generate_int64_configGenerateInt64
generate_last_name_configGenerateLastName
generate_sha256hash_configGenerateSha256Hash
generate_ssn_configGenerateSSN
generate_state_configGenerateState
generate_street_address_configGenerateStreetAddress
generate_string_phone_number_configGenerateStringPhoneNumber
generate_string_configGenerateString
generate_unixtimestamp_configGenerateUnixTimestamp
generate_username_configGenerateUsername
generate_utctimestamp_configGenerateUtcTimestamp
generate_uuid_configGenerateUuid
generate_zipcode_configGenerateZipcode
transform_e164_phone_number_configTransformE164PhoneNumber
transform_first_name_configTransformFirstName
transform_float64_configTransformFloat64
transform_full_name_configTransformFullName
transform_int64_phone_number_configTransformInt64PhoneNumber
transform_int64_configTransformInt64
transform_last_name_configTransformLastName
transform_phone_number_configTransformPhoneNumber
transform_string_configTransformString
passthrough_configPassthrough
nullconfigNull
user_defined_transformer_configUserDefinedTransformerConfig
generate_default_configGenerateDefault
transform_javascript_configTransformJavascript
generate_categorical_configGenerateCategorical
transform_character_scramble_configTransformCharacterScramble
generate_javascript_configGenerateJavascript
generate_country_configGenerateCountry
transform_pii_text_configTransformPiiTextNeosyncCloud/Enterprise only transformer for anonymizing PII Text
generate_business_name_configGenerateBusinessName
generate_ip_address_configGenerateIpAddress
transform_uuid_configTransformUuid

UpdateUserDefinedTransformerRequest

NameTypeDescription
transformer_idstringThe unique identifier of the user defined transformer to update.
namestringThe name of the user defined transformer.
descriptionstringThe description of the user defined transformer.
transformer_configTransformerConfigThe configuration of the user defined transformer.

UpdateUserDefinedTransformerResponse

NameTypeDescription
transformerUserDefinedTransformer

UserDefinedTransformer

NameTypeDescription
idstringThe unique identifier of the user defined transformer.
namestringThe name of the user defined transformer.
descriptionstringThe description of the user defined transformer.
data_typeTransformerDataType@deprecated: data_type is deprecated and will be removed in a future release. Use data_types instead.
sourceTransformerSourceThe source of the user defined transformer.
configTransformerConfigThe configuration of the user defined transformer.
created_atgoogle.protobuf.TimestampThe date and time the user defined transformer was created.
updated_atgoogle.protobuf.TimestampThe date and time the user defined transformer was last updated.
account_idstringThe unique identifier of the account that owns the user defined transformer.
data_typesrepeated TransformerDataTypeThe data types supported by the user defined transformer. This is currently calculated based off the origin system transformer.

UserDefinedTransformerConfig

NameTypeDescription
idstringThe id of the user defined transformer. When used in a job mapping, this will be used to reference the underlying system transformer along with it's configuration that was chosen.

ValidateUserJavascriptCodeRequest

NameTypeDescription
codestringThe user provided javascript code that will be validated.

ValidateUserJavascriptCodeResponse

NameTypeDescription
validbool

ValidateUserRegexCodeRequest

NameTypeDescription
user_provided_regexstringThe user provided regex to validate

ValidateUserRegexCodeResponse

NameTypeDescription
validboolWhether the regex is valid

Enums

GenerateEmailType

NameNumberDescription
GENERATE_EMAIL_TYPE_UNSPECIFIED0Unspecified defaults to uuid v4
GENERATE_EMAIL_TYPE_UUID_V41Generates a random uuid v4 (without hyphens) on the left side of the @. This has all the uniqueness guarantees of the standard uuid v4, along with randomized domains.
GENERATE_EMAIL_TYPE_FULLNAME2Generates a full name (firstnamelastname) on the left side of the @. This has high cardinality but is not guaranteed to be unique.

GenerateIpAddressType

NameNumberDescription
GENERATE_IP_ADDRESS_TYPE_UNSPECIFIED0Unspecified defaults to ipv4
GENERATE_IP_ADDRESS_TYPE_V4_PUBLIC1Generates a random ipv4_public
GENERATE_IP_ADDRESS_TYPE_V4_PRIVATE_A2Generates a random ipv4_private_a
GENERATE_IP_ADDRESS_TYPE_V4_PRIVATE_B3Generates a random ipv4_private_b
GENERATE_IP_ADDRESS_TYPE_V4_PRIVATE_C4Generates a random ipv4_private_c
GENERATE_IP_ADDRESS_TYPE_V4_LINK_LOCAL5Generates a random ipv4_linklocal
GENERATE_IP_ADDRESS_TYPE_V4_MULTICAST6Generates a random ipv4_multicast
GENERATE_IP_ADDRESS_TYPE_V4_LOOPBACK7Generates a random ipv4_loopback
GENERATE_IP_ADDRESS_TYPE_V68Generates a random ipv6

InvalidEmailAction

Action enumeration that dictates what the transform_email transformer should do in the case that it encounters an email that does not conform to RFC 5322

NameNumberDescription
INVALID_EMAIL_ACTION_UNSPECIFIED0Unspecified defaults to reject.
INVALID_EMAIL_ACTION_REJECT1If an invalid email is encountered, the transformer will reject the transformation and exit.
INVALID_EMAIL_ACTION_NULL2The result of the email will be null. This should only be specified if the destination column is also nullable.
INVALID_EMAIL_ACTION_PASSTHROUGH3The result of the email will be identical to the original value, unmodified.
INVALID_EMAIL_ACTION_GENERATE4The result of the email will be a brand new generated email. This is the same result as the generate_email transformer.

PiiAnonymizer.Hash.HashType

NameNumberDescription
HASH_TYPE_UNSPECIFIED0If unspecified, defaults to md5
HASH_TYPE_MD51The MD5 Hash Algorithm
HASH_TYPE_SHA2562The SHA256 Hash Algorithm
HASH_TYPE_SHA5123The SHA512 Hash Algorithm

SupportedJobType

NameNumberDescription
SUPPORTED_JOB_TYPE_UNSPECIFIED0
SUPPORTED_JOB_TYPE_SYNC1Determines if a transformer is supported for a sync job.
SUPPORTED_JOB_TYPE_GENERATE2Determines if a transformer is supported for a generate job.

TransformerDataType

NameNumberDescription
TRANSFORMER_DATA_TYPE_UNSPECIFIED0
TRANSFORMER_DATA_TYPE_STRING1
TRANSFORMER_DATA_TYPE_INT642
TRANSFORMER_DATA_TYPE_BOOLEAN3
TRANSFORMER_DATA_TYPE_FLOAT644
TRANSFORMER_DATA_TYPE_NULL5
TRANSFORMER_DATA_TYPE_ANY6
TRANSFORMER_DATA_TYPE_TIME7
TRANSFORMER_DATA_TYPE_UUID8

TransformerSource

NameNumberDescription
TRANSFORMER_SOURCE_UNSPECIFIED0
TRANSFORMER_SOURCE_PASSTHROUGH1
TRANSFORMER_SOURCE_GENERATE_DEFAULT2
TRANSFORMER_SOURCE_TRANSFORM_JAVASCRIPT3
TRANSFORMER_SOURCE_GENERATE_EMAIL4
TRANSFORMER_SOURCE_TRANSFORM_EMAIL5
TRANSFORMER_SOURCE_GENERATE_BOOL6
TRANSFORMER_SOURCE_GENERATE_CARD_NUMBER7
TRANSFORMER_SOURCE_GENERATE_CITY8
TRANSFORMER_SOURCE_GENERATE_E164_PHONE_NUMBER9
TRANSFORMER_SOURCE_GENERATE_FIRST_NAME10
TRANSFORMER_SOURCE_GENERATE_FLOAT6411
TRANSFORMER_SOURCE_GENERATE_FULL_ADDRESS12
TRANSFORMER_SOURCE_GENERATE_FULL_NAME13
TRANSFORMER_SOURCE_GENERATE_GENDER14
TRANSFORMER_SOURCE_GENERATE_INT64_PHONE_NUMBER15
TRANSFORMER_SOURCE_GENERATE_INT6416
TRANSFORMER_SOURCE_GENERATE_RANDOM_INT6417
TRANSFORMER_SOURCE_GENERATE_LAST_NAME18
TRANSFORMER_SOURCE_GENERATE_SHA256HASH19
TRANSFORMER_SOURCE_GENERATE_SSN20
TRANSFORMER_SOURCE_GENERATE_STATE21
TRANSFORMER_SOURCE_GENERATE_STREET_ADDRESS22
TRANSFORMER_SOURCE_GENERATE_STRING_PHONE_NUMBER23
TRANSFORMER_SOURCE_GENERATE_STRING24
TRANSFORMER_SOURCE_GENERATE_RANDOM_STRING25
TRANSFORMER_SOURCE_GENERATE_UNIXTIMESTAMP26
TRANSFORMER_SOURCE_GENERATE_USERNAME27
TRANSFORMER_SOURCE_GENERATE_UTCTIMESTAMP28
TRANSFORMER_SOURCE_GENERATE_UUID29
TRANSFORMER_SOURCE_GENERATE_ZIPCODE30
TRANSFORMER_SOURCE_TRANSFORM_E164_PHONE_NUMBER31
TRANSFORMER_SOURCE_TRANSFORM_FIRST_NAME32
TRANSFORMER_SOURCE_TRANSFORM_FLOAT6433
TRANSFORMER_SOURCE_TRANSFORM_FULL_NAME34
TRANSFORMER_SOURCE_TRANSFORM_INT64_PHONE_NUMBER35
TRANSFORMER_SOURCE_TRANSFORM_INT6436
TRANSFORMER_SOURCE_TRANSFORM_LAST_NAME37
TRANSFORMER_SOURCE_TRANSFORM_PHONE_NUMBER38
TRANSFORMER_SOURCE_TRANSFORM_STRING39
TRANSFORMER_SOURCE_GENERATE_NULL40
TRANSFORMER_SOURCE_GENERATE_CATEGORICAL42
TRANSFORMER_SOURCE_TRANSFORM_CHARACTER_SCRAMBLE43
TRANSFORMER_SOURCE_USER_DEFINED44
TRANSFORMER_SOURCE_GENERATE_JAVASCRIPT45
TRANSFORMER_SOURCE_GENERATE_COUNTRY46
TRANSFORMER_SOURCE_TRANSFORM_PII_TEXT47NeosyncCloud/Enterprise only transformer for anonymizing PII Text
TRANSFORMER_SOURCE_GENERATE_BUSINESS_NAME48
TRANSFORMER_SOURCE_GENERATE_IP_ADDRESS49
TRANSFORMER_SOURCE_TRANSFORM_UUID50

Services

TransformersService

Service that manages system and user defined transformers

GetSystemTransformers

MethodGetSystemTransformers
RequestGetSystemTransformersRequest
ResponseGetSystemTransformersResponse
DescriptionGet a list of all system transformers

GetSystemTransformerBySource

MethodGetSystemTransformerBySource
RequestGetSystemTransformerBySourceRequest
ResponseGetSystemTransformerBySourceResponse
DescriptionGet a system transformer by its source

GetUserDefinedTransformers

MethodGetUserDefinedTransformers
RequestGetUserDefinedTransformersRequest
ResponseGetUserDefinedTransformersResponse
DescriptionGet a list of all user defined transformers

GetUserDefinedTransformerById

MethodGetUserDefinedTransformerById
RequestGetUserDefinedTransformerByIdRequest
ResponseGetUserDefinedTransformerByIdResponse
DescriptionGet a user defined transformer by its id

CreateUserDefinedTransformer

MethodCreateUserDefinedTransformer
RequestCreateUserDefinedTransformerRequest
ResponseCreateUserDefinedTransformerResponse
DescriptionCreate a user defined transformer

DeleteUserDefinedTransformer

MethodDeleteUserDefinedTransformer
RequestDeleteUserDefinedTransformerRequest
ResponseDeleteUserDefinedTransformerResponse
DescriptionDelete a user defined transformer

UpdateUserDefinedTransformer

MethodUpdateUserDefinedTransformer
RequestUpdateUserDefinedTransformerRequest
ResponseUpdateUserDefinedTransformerResponse
DescriptionUpdate a user defined transformer

IsTransformerNameAvailable

MethodIsTransformerNameAvailable
RequestIsTransformerNameAvailableRequest
ResponseIsTransformerNameAvailableResponse
DescriptionCheck if a transformer name is available in your account

ValidateUserJavascriptCode

MethodValidateUserJavascriptCode
RequestValidateUserJavascriptCodeRequest
ResponseValidateUserJavascriptCodeResponse
DescriptionValidate user provided javascript code before saving it to a user defined transformer or within a job mapping

ValidateUserRegexCode

MethodValidateUserRegexCode
RequestValidateUserRegexCodeRequest
ResponseValidateUserRegexCodeResponse
DescriptionValidate user provided regex code before saving it to a user defined transformer

GetTransformPiiEntities

MethodGetTransformPiiEntities
RequestGetTransformPiiEntitiesRequest
ResponseGetTransformPiiEntitiesResponse
DescriptionRetrieve a list of available Pii entities for use with the TransformPiiText transformer