Describe the bug
When row_dq rules are configured for a table containing dots, Spark Expectations will automatically resolve all columns, even when it's not in the row_dq rules itself.
To Reproduce
Steps to reproduce the behavior:
- Go to your rules table
- Append a rule with type row_dq and target a table containing dots in the column names
Expected behavior
A clear and concise description of what you expected to happen.
It should automatically escape the column names using backticks.
Logs
error occurred while processing spark expectations error occurred
while executing func_process error occurred
while taking action on given rules [UNRESOLVED_COLUMN.WITH_SUGGESTION]
A column or function parameter
with name `event`.`actionCode` cannot be resolved.
Additional context
Spark Expectations version: v2.6.1
Describe the bug
When row_dq rules are configured for a table containing dots, Spark Expectations will automatically resolve all columns, even when it's not in the row_dq rules itself.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
It should automatically escape the column names using backticks.
Logs
Additional context
Spark Expectations version: v2.6.1