NEWS

Ora2Pg v24.3 released!, March 29 2024

This release fix several issues reported since last release and adds some new features and improvements.

  * Add option control the "prefetch" used by oracle_fdw COPY/INSERT
    Prior to this change Ora2Pg uses the default "prefetch" of oracle_fdw,
    which at the time of writing is 50. Allowing this to be controlled by an
    Ora2Pg configuration/option gives the option of increased performance at
    the cost of some additional memory on the PostgreSQL side. Thanks to Martin
    Nash for the patch.

  * Modify the behavior of triggers export with EXPORT_INVALID. It used to
    apply to ENABLED or DISABLED triggers instead of real VALID or INVALID
    triggers. Export of INVALID triggers will be controlled by EXPORT_INVALID
    like others objects like functions, packages, etc. This mean that disabled
    triggers that are valid will be exported by default now, this was not
    the case before. Thanks to dcgadmin for the feature request.

  * Add new configuration directive PGTT_NOSUPERUSER. By default the pgtt
    extension is loaded using the superuser privilege when EXPORT_GTT is
    activated. Enabled it if you run the SQL scripts generated using a non
    superuser user. It will use:
    
        LOAD '$libdir/plugins/pgtt';
    
    instead of default:
    
        LOAD 'pgtt';
    
    Thanks to Simon Martin for the feature request.

Here is the full list of changes and acknowledgements:

  - Fix Inf replacement that must only be done with numeric datatype. Thanks to
    gael-efluid for the report.
  - Fix some replacement of OUTER JOIN (+). Thanks to Carens Kurniawan Wijaya
    for the report.
  - Fix schema filter with test function count. Thanks to dcgadmin for the
    report.
  - Fix TEST count objects when a table name is modified. Thanks to korolan
    for the report.
  - Fix issue with multi style comments. Thanks to newtglobal.com for the patch.
  - Fix documentation about EXPORT_INVALID to precise that it also concern
    disabled triggers. Thanks to dcgadmin for the report.
  - Fix missing FOR EACH clause in trigger export after a regression introduced
    by commit fb6b0ad. Thanks to Carens Kurniawan Wijaya for the report.
  - Fix data export for table with a geometry column. Thanks to ruralqiu for the
    report.
  - Revert changes introduced by commit fc7008c, for some obscurs Oracle reasons
    the DBA_SDO_GEOM_METADATA doesn't always exist.
    Always uses ALL_SDO_GEOM_METADATA instead. Thanks to ruralqiu and
    Pierre3939 for the report.

Ora2Pg v24.2 released!, March 07 2024

This release fix several issues reported since last release and adds some new features and improvements.

  * Allow DATA_EXPORT_ORDER to take a filename at value to be able to
    give a custom table order. The file must contain the ordered list
    of the tables. One table per line in upper case for Oracle. Thanks
    to DataCloudGaze for the feature request.
  * Add progress bar when --oracle_speed is used to avoid waiting for
    the whole data export ends.
  * Add replacement of the BITAND function by the & operator
  * Add option -f, --format to set the output format for the reports.
    It can be html or json. Default to html. Thanks to mgole001 for the
    feature request.
  * Add automatic addition of the partition key to the primary key. Prefix
    all columns with the alias in the query to get data from Oracle.
  * Add information about MSSQL masked columns in SHOW_COLUMN.
  * Add information about columnstore and compression on MSCSQL tables with
    action SHOW_TABLE.
  * Add new configuration directive PARTITION_BY_REFERENCE to defined
    how to export Oracle partition by reference. Possible values are none,
    duplicate or the number of hash partitions to create.
    Value 'none' mean no translation and export of partition by reference
    like before. Value 'duplicate' will duplicate the referenced column
    in the partitioned table and apply the same partitioning from the
    referenced table to the partitioned table. If the value is a number,
    the table will be partitioned with the HASH method using the value
    as the modulo. For example if you set it to 4 it will create 4 HASH
    partitions.
    Default is none to not export the partitions by reference definition.

Here is the full list of changes and acknowledgements:

  - Fix default values in MSSQL function declaration and missing END keyword.
  - Fix parsing of MSSQL function with a single query. Thanks to Saravanan Newt
    for the report.
  - Fix negative Oracle decimal when comparing data. Thanks to es99-24 for the
    report.
  - Fix typos in documentation. Thanks to Simon Martin for the report.
  - Avoid doubling the NAME keyword in the XMLELEMENT() function.
  - Remove clause "REFERENCING OLD AS OLD NEW AS NEW" that is useless and throw
    an error in PostgreSQL
  - Fix DSN for MSSQL in autogenerated configuration file using --init_project.
  - Fix parsing of REFERENCING clause in trigger. Thanks to Carens Kurniawan
    Wijaya for the report.
  - Fix case where ALL_TAB_COLUMNS and ALL_SDO_GEOM_METADATA was used instead
    of USER_TAB_COLUMNS. Thanks to rvanouter for the report.
  - Fix double quoting in partition by reference where clause.
  - Disallow setting of PARTITION_BY_REFERENCE to duplicate when FDW_SERVER
    is set.
  - Fix trigger export with duplicate FOR EACH clause when a REFERENCING clause
    is present. Thanks to Carens Kurniawan Wijaya for the report.
  - Fix typo in variable name in ora2pg_scanner
  - Fix export of default partition for MySQL. Thanks to Priyanshi Gupta for
    the report.
  - Fix TEST_DATA action with issues on foreign server and import foreign schema
    declaration. Thanks to Florent Jardin for the report.
  - Fix export of NOT NULL constraint with column replacement. Thanks to Florent
    Jardin for the report.
  - Prevent reading file ora2pg_stdout_locker when it does not exist. Thanks to
    Florent Jardin for the report.
  - Fix translation of timestamp with precision < 6. Thanks to Andrei Briukhov
    for the report.
  - Fix json_arrayagg with returning clause, type was not translated.
  - Fix data export for table partitioned by reference with duplicate method.
    The query to extract data on Oracle side add the duplicated column and
    perform the join with the referenced table using the FK definition.
  - Fix partition by LIST export with a useless cast to text. Thanks to
    Priyanshi Gupta for the report.
  - Added closing curly bracket for object details. Thanks to andreas42 for
    the patch.
  - Added quotes around value of "human days cost" and "migration level".
    Thanks to andreas42 for the patch.
  - Format object type detail output as JSON array. Thanks to andreas42 for
    the patch.
  - Fix variable declaration in previous commit
  - Fix MSSQL table export with nonexistent column auto_created in version
  bellow 2017. Thanks to Florent Jardin for the report.
  - Use ADD CONSTRAINT syntax to specify name of primary key. Thanks to Martin
    Karlgren for the patch.
  - Skip unwanted work on PG database when option --oracle_speed is enabled,
    especially drop of constraints. Thanks to John Tian for the report.
  - Remove trailing ); from primary key statements when reading from file.
    Thanks to Martin Karlgren for the patch.
  - Fix option in CREATE USER MAPPING for export of DBLINK.
  - Fix translation of MSSQL floating point datatype to use float(n) notation.
  - Exclude from assessment objects in Oracle bin.
  - Add Oracle package HTP and HTF to migration assessment.
  - Fix MSSQL foreign key export with multiple columns referenced.
  - Fix MSSQL export of unique constraints that was merging columns of several
    unique constraints for the same table into a single constraint.
  - Fix case of MSSQL datetime default value 0 that must be converted to
    '1900-01-01 00:00:00'
  - Add rewrite of MSSQL getutcdate() function.
  - Fix MSSQL index type and add compression + columnstore information
  - Add clause IF NOT EXIST to all CREATE EXTENSION calls
  - Fix MSSQL bit data migration through tds_fdw, it is already exported
    as boolean by the fdw.
  - Fix duplicated indexes in MSSQL export.
  - Add export of MSSQL indexes with columns included (CREATE INDEX+INCLUDE).

Ora2Pg v24.1 released!, September 08 2023

This release fix several issues reported since last release and adds some new features and improvements.

  - Replace "set feedback off" by "\set QUIET on;" and "set pagesize 0" with
    "\pset pager off". Thanks to Martin Gerhardy for the suggestion.
  - Always add package name into search_path of packages functions. Thanks to
    janopha for the report.
  - Allow to specify a password file to set PG_PWD at PostgreSQL database
    connection.  If the specified file exists on the system, Ora2Pg will read
    the first line to get the password at each call of send_to_pgdb(). It can
    be useful in some situation where the password change during data migration.
    Thanks to Marius Hope for the feature request.
  - Added option --dump_as_json and fixed some json output errors. Thanks to
    Martin Gerhardy for the patch.

Here is the full list of changes and acknowledgements:

  - Fix schema prefixing of type created in packages stored procedures.
    Thanks to janopha for the report.
  - Fix perl function get_schema_condition() to use quote_ident() in generated
    SQL filters. Thanks to franxav06 for the report.
  - Attempt to better test the not null constraint count in Oracle.
  - Fix regression with PSQL_RELATIVE_PATH. Thanks to Ryan Taylor for the
    report.
  - Do not add the partition key to PK if DISABLE_PARTITION is enabled. Thanks
    to pavel-moskotin-db for the report.
  - Filter list of indexes on name instead of the GENERATED column because
    we are missing the ones that have been created automatically by the
    Automatic Indexing feature of Oracle 19c. Thanks to Franck Pachot for
    the report.
  - Fix incompleteness in GRANT action, add grant usage on schema to owner and
    users. Thanks to elexus for the report.
  - Apply missing --blob_to_lo to import_all.sh script.
  - Fix regression in MySQL hash partitions export. Thanks to Shubham Dabriwala
    for the report.
  - Fix Oracle INTERVAL data export with negative value. Thanks to shubham-yb
    for the report.
  - Fix double count of not null constraints for TEST action. Thanks to Simon
    Pane for the patch.
  - Fix replacement of Oracle sys_refcursor in function return type.
  - Fix replacement of SQL script setting from Oracle.
  - Make scripts executable. Thanks to Martin Gerhardy for the patch.
  - Exclude data pump SYS_EXPORT_SCHEMA_.* tables from Oracle export.
  - ora2pg: use env based shebang for perl. Thanks to Martin Gerhardy for
    the patch.
  - Add unsupported clause message for PRAGMA AUTONOMOUS_TRANSACTION when it is
    not rewritten by Ora2Pg. Thanks to Martin Gerhardy for the patch.
  - Allow schema specific definitions of partitioning columns. Thanks to Martin
    Gerhardy for the patch.
  - Fixed invalid variable name in read_grant_from_file. Thanks to Martin
    Gerhardy for the patch.
  - Fix not double quoted column in alter set not null column statement.
    Thanks to leonteq-reisg for the patch.
  - Bug fixes and special treatment for Types with body. Thanks to Martin
    Gerhardy for the patch.
  - Avoid redundant definition of the version. Thanks to Martin Gerhardy for
    the patch.
  - Fix a MySQL exception when the column type is ENUM for version < 5.7.
    Thanks to Code-UV for the report.

Ora2Pg v24.0 released!, July 05 2023

This major release adds support to migration of SQL Server database to PostgreSQL. It also fixes several issues reported since past height months and adds some new features and improvements.

  * Enable the use of ALLOW/EXCLUDE directive with SHOW_* reports and throw
    a fatal error if global filters in ALLOW/EXCLUDE are set.
  * Add replacement of DBMS_LOCK.SLEEP with pg_sleep
  * Split estimate cost details per function/procedure/and package function.
  * Add cmin, cmax, ctid to reserved keywords list.
  * Add cost for presence of ADD CONSTRAINT in PLSQL code. It needs constraint
    name stability.
  * Add CLOB_AS_BLOB configuration directive to treat CLOB as BLOB when
    exporting data. When enabled Ora2Pg will apply same behavior on CLOB
    than BLOB with BLOB_LIMIT setting. This could be useful if you have
    large CLOB data. Enabled by default. Thanks to Omar Mebarki for the patch.
  * Allow COPY and TABLE type to use the NULLIF construct. Thanks to Luke Davies
    for the patch.
  * Add new SEQUENCE_VALUES export type to export DDL to set the last values
    of sequences from current Oracle database last values like the following
    statements: ALTER SEQUENCE departments_seq START WITH 290;
    Thanks to sergey grinko for the feature request.
  * Add replacement of Oracle variable : varname into PG :'varname'.
  * Add SQL Server migration to Ora2Pg. Most of the SQL Server objects are
    supported as well as data export. Translation of the TSQL stored
    procedures to plpgsql is complicated because of the lack of statement
    separator in TSQL but as usual Ora2Pg is doing is best to do as much
    work as possible. Migration assessment is also possible with SQL Server
    database. There is some dedicated configuration directives added to
    ora2Pg.conf.
  * Add support to MySQL PARTITION BY KEY() with a translation to HASH
    partitioned table using the PK/UK definition of the table or the
    columns specified in the KEY() clause. Thanks to Shubham Dabriwala
    for the report.
  * Make EXPORT_INVALID configuration directive works with TRIGGER export.
    Until now disabled triggers were not exported, setting EXPORT_INVALID
    to 1 will force the export of disabled triggers. Thanks to chetank-yb
    for the report.
  * Add support of MySQL generated default value on update. For example:
      CREATE TABLE t1 (
        dt DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
      );
    Ora2Pg will translate this syntax into a trigger on the table to force
    the value of the column on an update event.
    Thanks to heysky for the report.
  * Add translation of ST_GEOMETRY data type to PostGis geometry datatype.
  * Replace ROWNUM in target list with a "row_number() over ()" clause. Thanks
    to Rui Pereira for the report.

New configuration directives:

  * Add configuration directive ST_GEOMETRYTYPE_FUNCTION to be able to set the
    function to use to extract the geometry type from a ST_Geometry column.
    Default: ST_GeometryType, example it should be set to sde.ST_GeometryType
    for ArcSDE. Thanks to Albert88373 for the report.
  * Add four new configuration directive to be able to change or prefix the
    functions used to extract information from ST_Geometry object and values.
    - ST_SRID_FUNCTION: Oracle function to use to extract the srid from
      ST_Geometry meta information. Default: ST_SRID, for example it should be
      set to sde.st_srid for ArcSDE.
    - ST_DIMENSION_FUNCTION: Oracle function to use to extract the dimension
      from ST_Geometry meta information. Default: ST_DIMENSION, for example it
      should be set to sde.st_dimention for ArcSDE.
    - ST_ASBINARY_FUNCTION: Oracle function to used to convert an ST_Geometry
      value into WKB format. Default: ST_ASBINARY, for example it should be set
      to sde.st_asbinary for ArcSDE.
    - ST_ASTEXT_FUNCTION: Oracle function to used to convert an ST_Geometry
      value into WKT format. Default: ST_ASTEXT, for example it should be set
      to sde.st_astext for ArcSDE.
    Thanks to Albert88373 for the report.
  * Add INSERT_ON_CONFLICT configuration directive. When enabled this instruct
    Ora2Pg to add an ON CONFLICT DO NOTHING clause to all INSERT statements
    generated for this type of data export. Thanks to Clemens Rieder for the
    feature request.

Backward compatibility:

  * Change the behavior of CASE_INSENSITIVE_SEARCH to allow the use of a
    collation instead of the citext extension. To disable the feature the
    value none can be used. If the migration is not MSSQL this feature is
    disabled.
  * Remove PREFIX_PARTITION configuration directive, it is now replaced by
    the RENAME_PARTITION directive. Previous behavior was to construct the
    partition name from the table name, the partition name and the sub
    partition name if any. The problem is that we often reach the max length
    for an object name and this leads to duplicate partition name. Now, when
    RENAME_PARTITION is enabled the partition tables will be renamed
    following rules:
        _part
    where "pos" is the partition number. For subpartition this is:
       _part_subpart
    If this is partition/subpartition default:
        _part_default
        _part_subpart_default
    This change will break backward comaptibilty, if PREFIX_PARTITION is
    still set, it will simply enable RENAME_PARTITION.
  * Set START value to MINVALUE when a sequence is cycled and that the START
    value is upper that MAXVALUE. Thanks to Shane Borden for the report.

Here is the full list of changes and acknowledgements:

  - Fix MODIFY_STRUCT that was not working with MySQL. Thanks to Code-UV for
    the report.
  - Fix license string in Makefile.PL. Thanks to RodRaen for the report.
  - Do not remove non alphanumeric character in index name. Thanks to gwidt
    for the report.
  - Reorder trigger event when the update of column is not the last one. Thanks
    to tayalarun1 for the report.
  - Fix export of MySQL function containing special characters and white spaces
    in names. Thanks to Shubham Dabriwala for the report.
  - Fix grant export for partitions. Thanks to elexus for the report.
  - Add some other transformation for sqlplus/psql scripts.
  - Remove comma as possible separator for values in DEFINED_PK, it was
    preventing the use of a function with multiple parameters.
  - Fix export of geometry tables when PG_SCHEMA is set.
  - Add rewriting of some sqlplus settings to psql settings.
  - Fix TABLESPACE export for partitioned tables. Thanks to elexus for the
    report.
  - Fix for Issue #1637. Thanks to Simon Pane for the patch.
  - Fix typo in --init_project directories tree generation for sequences
    values.
  - Fix alias in view target list for function call without alias defined in
    MySQL export. Thanks to Shubham Dabriwala for the report.
  - Fix Mysql procedure export when a datatype with precision is used in
    parameter list. Thanks to Shubham Dabriwala for the report.
  - Fix collation on string default values. Thanks to Shubham Dabriwalafor
    the report.
  - Exclude recycle bin object from ALL_TAB_COLUMNS lookup. Thanks to Dave
    Betterton for the report.
  - Fix data types translation (TINYINT|SMALLINT|MEDIUMINT|INTEGER|BIGINT|INT)
    for MySQL table export. Thanks to Shubham Dabriwala for the report.
  - Do not export synonym destination table with table_owner when EXPORT_SCHEMA
    is disabled. Thanks to Priyanshi Gupta for the report.
  - Fully qualify calls to get_sequence_last_values() when PG_SCHEMA is set.
    Thanks to Marius Hope for the report.
  - Fix regression on exporting view as table when VIEW_AS_TABLE contains
    regexp. Thanks to Neil Bower for the report.
  - Fix missing execution of initial command statements at start of TEST_DATA
    action and on both side, those applying to source and destination. Thanks
    to Petter Jacobsen for the report.
  - Fix script to get sequence last value with TEST action. Thanks to franxav06
    for the patch.
  - Prepend PERFORM before call to DBMS_OUTPUT.* when USE_ORAFCE is enabled.
  - Disable USE_ORAFCE when export type is SHOW_REPORT.
  - Extending the enhancement in Pull Request #1621 to the Oracle_FDW user
    mapping. Thanks to Simon Pane for the patch.
  - Changed prefix string to "DIFF:" in test report. Thanks to Simon Pane for
    the patch.
  - Fix cases where %ROWCOUNT was not correctly replaced. Thanks to Rui Pereira
    for the report.
  - Fix parsing of ORACLE_DSN when creating foreign server in COPY mode. Thanks
    to Luke Davies for the report.
  - Fix for Issue #1622, #1627. Thanks to Simon Pane for the patch.
  - Fix index creation with DESC order in COPY action when DROP_INDEXES is
    enabled. Thanks to Luke Davies for the report.
  - Fix for Issue #1610, #1612, #1617 and #1381. Thanks to Simon Pane for the
    patch.
  - Fix typo in sqlnet.ora name (was sqlnet.or). Thanks to Martin Nash for the
    patch.
  - Fix data export, REPLACE_QUERY was not applied. Thanks to Bachev Constantin
    for the report.
  - Fix call to replace_sys_context().
  - Fix timestamp(n) data type translation.
  - Remove use of column GENERATION_EXPRESSION for MySQL version < 5.7.0. Thanks
    to Hans Choi for the report.
  - Fix conversion of DATE datatype to timestamp(0) instead of timestamp. Thanks
    to Akhil Reddy for the report.
  - Add NVARCHAR/NCHAR defaut convertion data types to DATA_TYPE configuration
    directive in ora2pg.conf comments. Thanks to Akhil Reddy for the report.
  - Rename method _get_partitions_type function into _get_partitions_list.
  - Fix synonym export when no schema information is available.
  - Fix support of REFERENCING clause in triggers.
  - Fix partition output file renaming with new RENAME_PARTITION directive.
    Thanks to Rahul Barigidad for the report.
  - Fix export of the ROWNUM clause when there is a variable.
  - Fix sprintf placeholders in geometry queries.
  - Fix some others issues with row count report.
  - Fix row count with destination schema and when the PostgreSQL table
    doesn't exist.
    Thanks to bizen-ya for the report.
  - Fix tests comparison with the different settings of EXPORT_SCHEMA,
    SCHEMA and PG_SCHEMA. Thanks to Marius Hope and bizen-ya for the
    report.
  - Fix St_AsText() call for MySQL data extraction.
  - Add column count comparison for MySQL
  - Export multi column partition by list as an expression with concat
    operator. Multi column partition by list is not supported by PostgreSQL.
  - Fix creation of non existant indexes on partition. Thanks to Shubham
    Dabriwala for the report.
  - Fix MySQL function export when there is no BEGIN clause. Thanks to
    Shubham Dabriwala for the report.
  - Fix MySQL export of unsigned numeric. Thanks to Shubham Dabriwala for
    the report.
  - Fix MySQL output with wrong synthax for JOIN without ON clause. Thanks
    to Shubham Dabriwala for the report.
  - Fix virtual column export. Thanks to Rafal Hollins for the report.
  - Fix index creation on partition with no columns for MySQL export.
    Thanks to Shubham Dabriwala for the report.
  - Fix export of MySQL auto_increment when PG_INTEGER_TYPE is disabled.
    Thanks to Shubham Dabriwala for the report.
  - Fix MySQL subpartition export. Thanks to Sanyam Singhal for the report.
  - Move any INTO clause in CONNECT BY query to the final SELECT on the
    resulting CTE. Thanks to taptarap for the report.
  - Fix translation of MySQL curtime() function in default values. Thanks
    to Shubham Dabriwala for the report.
  - Fix possible "Nested quantifiers in regex" error when exporting package
    with package name containing regex special characters. Thanks to durandm70
    for the report.
  - Fix documentation about use of unique key for ORACLE_COPY.
  - Fix extra comma at end of a CHECK contraint. Thanks to Shubham Dabriwala
    for the report.
  - Always add DROP TYPE statements with package export even if DROP_IF_EXISTS
    is not enabled. Thanks to Rui Pereira for the report.
  - Fix default value of simple dot in MySQL export. Thanks to Shubham
    Dabriwala for the report.
  - Fix regression in data type translation after fix on unsigned numeric type.
    Thanks to Shubham Dabriwala for the report.

Ora2Pg v23.2 released!, October 08 2022

This release fix several issues reported since past height months and adds some new features and improvements.

  * Add export of MySQL KEY and LINEAR KEY partitioning, translated as HASH
    partitioning.
  * Allow export of object with dollar sign is his name.
  * Add export of CHECK constraints for MySQL >= 8.0.
  * Add Functional/Expression indexes export from MYSQL.
  * Add export of MySQL virtual column. Thanks to Shubham Dabriwala for the
  * Remove scale and precision of a numeric if the scale is higher than the
    precision. PostgreSQL does not support decimal/numeric attributes where
    the scale is explicitly higher than the precision.
  * Add command line option --drop_if_exists to add statement to drop objects
    before creation if it exists. It corresponds to the DROP_IF_EXISTS
  * Add option -C | --cdc_file to be able to change the name of the default file
    used to store/read SCN per table during export. Default is TABLES_SCN.log in
    the current directory. This is the file written by the --cdc_ready option.
  * Add multiprocess to count rows in PostgreSQL tables (TEST_COUNT) using -P
    command line option.
  * Add support to PostgreSQL 14 procedure with out parameters.
  * Set default PostgreSQL database target version to 14.

New configuration directives:

  * Add configuration directive MVIEW_AS_TABLE and command line option
    --mview_as_table to set which materialized view to export as table.
    By default none. Value must be a list of materialized view name or
    regexp separated by space or comma. If the object name is a materialized
    view and the export type is TABLE, the view will be exported as a create
    table statement. If export type is COPY or INSERT, the corresponding data
    will be exported.
  * Add configuration variable FDW_IMPORT_SCHEMA to rename the schema where
    foreign tables for data migration will be created. If you use several
    instances of ora2pg for data migration through the foreign data wrapper,
    you might need to change the name of the schema for each instance.
    Default: ora2pg_fdw_import
  * Add TRANSFORM_VALUE configuration directive to apply an expression when
    retrieving data from Oracle. For example:
        TRANSFORM_VALUE   CLOB_TABLE[CHARDATA:translate("CHARDATA", chr(0), ' ')]
    to replace all Oracle char(0) in a string by a space character.
  * Add EXCLUDE_COLUMNS configuration directive. Unlike MODIFY_STRUCT
    that is used to redefine a table structure, this directive allow
    to specify a list of columns per table that must be excluded from
    the export. For example:
        EXCLUDE_COLUMNS        T1(nocol1,nocol2) T2(nocol1,nocol2)
  * Add new configuration directive EXPORT_GTT to export Oracle Global Temporary
    Table using syntax recognized by the pgtt extension. For more information see
    https://github.com/darold/pgtt Default is to not export global temporary
    table as they are not supported natively by PostgreSQL.
  * Add new configuration option NO_EXCLUDED_TABLE. By default Ora2Pg exclude
    from export some Oracle "garbage" tables that should never be part of an
    export. This behavior generates a lot of REGEXP_LIKE expressions which are
    slowing down the export when looking at tables. To disable this behavior
    enable this directive, you will have to exclude or clean up later by
    yourself the unwanted tables. The regexp used to exclude the table are
    defined in the array @EXCLUDED_TABLES in lib/Ora2Pg.pm. Note this is behavior
    is independent to the EXCLUDE configuration directive.

Backward compatibility:

  * Force rewrite of all invalid date starting with zero year 0000 into 1970-01-01
    when it is a default value and NULL for data. Old behavior was to only replace
    0000-00-00 date.
  * Until now there was a lot of untranslated call to TRUNC(date) because
    Ora2Pg is unable to detect that the parameter is a date or a number.
    The problem is that Oracle has TRUNC(number) too and Ora2Pg try to not
    apply the transformation if there is a doubt. In most of the migration
     have met very few TRUNC(number) so now all call to TRUNC()
    will be converted to date_trunc(). There must be false positive rewrite
    but this should be far less work than the actual situation.

For a complete list of change see https://github.com/darold/ora2pg/blob/master/changelog

Ora2Pg v23.1 released!, February 10 2022

This release fix several issues reported since past four months and adds some new major features and improvements.

  * Add use of greatest/least functions from new version of Orafce when
    required to return NULL on NULL input like Oracle.
  * ALLOW and EXCLUDE configuration values can now be read from a file.
    Use -a filename or -e filename to specify the list of tables that need
    to be filtered. This is useful if you have a lot of table to filter.
  * Add possibility to use of System Change Number (SCN) for data export or
    data validation by providing a specific SCN.  It can be set at command
    line using the -S or --scn option. You can give a specific SCN or if you
    want to use the current SCN at first connection time set the value to
    'current'. To use this last case the connection user must have the role
    "SELECT ANY DICTIONARY" or "SELECT_CATALOG_ROLE", the current SCN is
    looked at the v$database view.
    Example of use:
        ora2pg -c ora2pg.conf -t COPY --scn 16605281
    This adds the following clause to the query used to retrieve data for example:
        AS OF SCN 16605281
    You can also use th --scn option to use the Oracle flashback capability by
    specifying a timestamp expression instead of a SCN. For example:
        ora2pg -c ora2pg.conf -t COPY --scn "TO_TIMESTAMP('2021-12-01 00:00:00', 'YYYY-MM-DD HH:MI:SS')"
    This will add the following clause to the query used to retrieve data:
        AS OF TIMESTAMP TO_TIMESTAMP('2021-12-01 00:00:00', 'YYYY-MM-DD HH:MI:SS')
    or for example to only retrieve yesterday's data:
        ora2pg -c ora2pg.conf -t COPY --scn "SYSDATE - 1"
  * Add json output format to migration assessment. Thanks to Ted Yu for the patch.
  * Add new TO_CHAR_NOTIMEZONE configuration directive to remove any timezone
    information into the format part of the TO_CHAR() function. Disabled by default.
    Thanks to Eric Delanoe for the report.
    Note that the new default setting breaks backward compatibility, old behavior
    was to always remove the timezone part.
  * Add new configuration directive FORCE_IDENTITY_BIGINT. Usually identity
    column must be bigint to correspond to an auto increment sequence so
    Ora2Pg always force it to be a bigint. If, for any reason you want
    Ora2Pg to respect the DATA_TYPE you have set for identity column then
    disable this directive.
  * Add command line option --lo_import. By default Ora2Pg imports Oracle BLOB
    as bytea, the destination column is created
    using the bytea data type. If you want to use large object instead of bytea,
    just add the --blob_to_lo option to the ora2pg command. It will create the
    destination column as data type Oid and will save the BLOB as a large object
    using the lo_from_bytea() function. The Oid returned by the call to
    lo_from_bytea() is inserted in the destination column instead of a bytea.
    Because of the use of the function this option can only be used with actions
    SHOW_COLUMN, TABLE and INSERT. Action COPY is not allowed.
    If you want to use COPY or have huge size BLOB ( > 1GB) than can not be
    imported using lo_from_bytea() you can add option --lo_import to the
    ora2pg command. This will allow to import data in two passes:
    1) Export data using COPY or INSERT will set the Oid destination column
       for BLOB to value 0 and save the BLOB value into a dedicated file. It
       will also create a Shell script to import the BLOB files into the
       database using psql command \lo_import and to update the table Oid
       column to the returned large object Oid. The script is named
       lo_import-TABLENAME.sh
    2) Execute all scripts lo_import-TABLENAME.sh after setting the
       environment variables PGDATABASE and optionally PGHOST, PGPORT, PGUSER,
       etc. if they do not correspond to the default values for libpq.
       You might also execute manually a VACUUM FULL on the table to remove
       the bloat created by the table update.
    Limitation: the table must have a primary key, it is used to set the
    WHERE clause to update the Oid column after the large object import.
    Importing BLOB using this second method (--lo_import) is very slow so it
    should be reserved to rows where the BLOB > 1GB for all other rows use
    the option --blob_to_lo. To filter the rows you can use the WHERE
    configuration directive in ora2pg.conf.
  * Add command line option --cdc_ready to use current SCN per table when
    exporting data and register them into a file named TABLES_SCN.log This
    can be used for Change Data Capture (CDC) tools.
  * Allow to export only invalid objects when EXPORT_INVALID is set to 2
  * Disable per partition data export when a WHERE clause is define on the
    partitioned table or that a global WHERE clause is defined.

Backward compatibility:

Ora2Pg used to removr any timezone information from the TO_CHAR() format function. To recover this behavior set TO_CHAR_NOTIMEZONE to 1 in ora2pg.conf

For a complete list of change see https://github.com/darold/ora2pg/blob/master/changelog

Ora2Pg v23.0 released!, November 15 2021

This release fix several issues reported since past five months and adds some new major features and improvements.

  * Add new option --blob_to_lo that can be used to export BLOB as large
    objects. It can only be used with action SHOW_COLUMN, TABLE and INSERT.
    When used with TABLE action, the BLOB column will be translated into oid
    PostgreSQL data type. When used with the INSERT export action BLOB data
    will be store as large object in the pg_largeobjects table and the oid
    referencing this large object will be stored in the main table instead
    of a bytea.
    It is not possible to use oid with COPY because this feature use function
    lo_from_bytea() that stores the large object in the external table and
    returns the oid.
    This feature works with or without the use of oracle_fdw to import the
    data and option -J can be used to improve the speed of the INSERT import
    provide that there is a numeric unique key on the table.
    Thanks to rodiq for the feature request.
  * Add command line option -W | --where clause to set the WHERE clauses to
    apply to the Oracle query to retrieve data. It can be used multiple time.
    It will override the WHERE configuration directive if there is a global
    WHERE clause or the same table WHERE clause definition. Otherwise the
    clause will be appended.
  * Add data validation feature consisting in comparing data retrieved from a
    foreign table pointing to the source Oracle table and a local PostgreSQL
    table resulting from the data export. By default Ora2Pg will extract 10000
    rows from both side, you can change this value using DATA_VALIDATION_ROWS.
    When it is set to zero all rows of the tables will be compared.
    Data validation requires that the table has a primary key or unique index
    and that the key columns is not a LOB.
    Due to differences in sort behavior between Oracle and PostgreSQL, if the
    collation of unique key columns in PostgreSQL is not 'C', the sort order of
    is different compared to Oracle. In this case the data validation will fail.
    Ora2Pg will stop comparing two tables after 10 errors, result is dumped to
    an output file named data_validation.log.
  * Add DATA_VALIDATION_ORDERING configuration directive enabled by default.
    Order of rows between both sides are different once the data have been
    modified. In this case data must be ordered using a primary key or a
    unique index, that mean that a table without such object can not be
    compared. If the validation is done just after data import in mode single
    process and without any data modification the validation can be done on all
    tables without any ordering.
  * Add DATA_VALIDATION_ERROR to stop validating data from a table after a
    certain amount of row mismatch. Default is to stop after 10 rows
    validation errors.
  * Allow multiprocess for TEST_DATA action to validate data import. Use -P
    or PARALLEL_TABLES to set the number of parallel tables checked. Output
    is now done to a file named data_validation.log saved in the current
    directory.
  * Add replacement of UTL_RAW.CAST_TO_RAW with encode().
  * Add rewrite of XMLTYPE() with xmlparse(DOCUMENT convert_from(..., 'utf-8')).
  * Add VARCHAR_TO_TEXT configuration directive. By default VARCHAR2 without
    size constraint are tranlated into text PG data type. If you want to use
    varchar instead, disable this directive.
  * Add detection of XML function for migration assessment cost.
  * Add DBMS_RANDOM to the list of Oraclism handled by Orafce.
  * Add support to mysql_fdw foreign data wrapper to export data
    PostgreSQL tables. Thanks to Yoni Sade for the feature request.
  * Allow to transform all NUMBER(*,scale) to an other data type by a
    redefinition like NUMBER(*\,2):decimal in the DATA_TYPE configuration
    directive. Thanks to Florent Jardin for the patch.
  * Add information on how to use SSL encrypted connection to documentation.
  * Add TEST_COUNT action to just report the row count diff between Oracle and

Backward compatibility changes:

  - Add FORCE_PLSQL_ENCODING configuration directive. In previous version Ora2Pg
    was encoding all functions code to ut8, this is no more the case because it
    could result in double encoding. To recover the old behavior (not recommanded)
    enable this directive. Thanks to rynerisraid and lee-jongbeom for the report.
  - Change behavior regarding RAW columns. Now RAW(16) and RAW(32) columns or
    RAW columns with "SYS_GUID()" as default value are now automatically
    translated into uuid. Data will be automatically migrated as PostgreSQL
    uuid data type provided by the "uuid-ossp" extension. To recover the old
    behavior to export data as bytea whatever is the precision, the following
    must be set with DATA_TYPE configuration:  RAW(16):bytea,RAW(32):bytea

For a complete list of change see https://github.com/darold/ora2pg/blob/master/changelog

Ora2Pg v22.1 released!, July 2nd 2021

This is a maintenance release to extend the feature of data export through the oracle_fdw PostgreSQL extension to migration that use the public schema and do not preserve case.

There is also some other fixes:

  - Fix compile_schema() call that breaks valid function based indexes by
    adding compile_all => FALSE to DBMS_UTILITY.compile_schema().
    Thanks to Pawel Fengler for the patch.
  - Force foreign table for data export as readonly to avoid accidental
    write if import schema is not cleaned.
  - Fix data export to file not possible since last changes for oracle_fdw
    export. Thanks to Niels Jespersen for the report.

Ora2Pg v22.0 released!, June 28th 2021

This release fix several issues reported since past three months and adds some new features and improvements. I must thanks MigOps Inc who hire me to drive Oracle to PostgreSQL migrations and to develop Ora2Pg. It's been a long time that I was looking for such a company and it is an amazing gift for the 20 years of Ora2Pg. All improvements and new new features developed during my work at MigOps will be available in the public GitHub repository, here are the new ones.

  - Add export of data using oracle_fdw when FDW_SERVER is set and export
    type is COPY or INSERT. Multi-process using -P or -J is fully supported
    but option -j is useless in this case. Boolean transformation of some
    columns or data type is also supported. Actually, expect that it works
    just like data migration without oracle_fdw. This can improve the data
    migration speed from 30 to 40% especially for BLOB export.

  - Improve export performances with huge number of objects by avoiding join
    between Oracle catalog tables.

  - Set a maximum of assessment score for tables, indexes, sequences,
    partitions, global temporary table and synonym following the number of
    objects.

  - Add detection of XML functions to the assessment cost.

  - Allow to change the assessment cost unit value in the export_all.sh script
    when ora2pg is used with options --init_project and --cost_unit_value.

  - Remove pragma restrict_references from P/PSQL code, it is useless.

  - Add the oracle schema to search_path in SQL files generated and improve
    the migration assessment when USE_ORAFCE is enabled.

  - Apply ALLOW and EXCLUDED filtered stored procedures at package extraction
    level. Previous this patch there was no way to not export some package
    functions or to exclude them from assessment.

  - Add new tests to check sequences last values and number of identity columns
    in both side.

  - Apply ALLOW/EXCLUDE without object to table object by default in TEST
    action.

New configuration directives:

  - Add ORACLE_FDW_TRANSFORM configuration directive to apply a transformation
    to a column when exporting data. Value must be a semicolon separated list of
        TABLE[COLUMN_NAME, >replace code in SELECT target list<]
    For example to replace string 'Oracle' by 'PostgreSQL' in a varchar2 column
    use the following.
        ERROR_LOG[DBMS_TYPE:regexp_replace("DBMS_TYPE",'Oracle','PostgreSQL')]
    Thanks to MigOps for the patch.

  - Add DROP_IF_EXISTS configuration directive to add a statement
    "DROP >OBJECT< IF EXISTS" before creating the object. Can be
    useful in an iterative work. Default is disabled. Thanks to
    dherzhau for the feature request.

Backward compatibility:

There is a backward compatibility issue with old configuration files where FDW_SERVER is set by default. This directive was not used when exporting data, this is not the case anymore as it instruct Ora2Pg to use the given foreign server to use oracle_fdw to migrate the data.

For a complete list of change see https://github.com/darold/ora2pg/blob/master/changelog

Ora2Pg v21.1 released!, April 1st 2021

This release fix several issues reported since past six months and adds some new features and improvements.

  * Now that Orafce 3.15.0 has a definition for the REGEXP_* function,
    makes the translation optional to USE_ORAFCE directive.
  * Add set application name in connection to Oracle/MySql/PostgreSQL.
  * Add translation of REGEXP_COUNT() and change assessment cost.
  * Rewrite the way REGEXP_LIKE() is translated into regexp_match to
    support modifiers. This rewrite also fix default behavior between
    Oracle and PostgreSQL.
  * Replace DBMS_LOB.GETLENGTH() by PostgreSQL octet_length() function.
  * Add types correspondences for VARCHAR2 and NVARCHAR2 in DATA_TYPE
    configuration directive.
  * Add autodetection and support of geometry type, srid and dimension
    for ArcGis geometries.
  * Add conversion of default value in function parameters.
  * Add -u | --unit option to ora2pg_scanner to be able to set the
    migration cost unit value globally.
  * Replace DBMS_LOB.SUBSTR() by SUBSTR()
  * Remove TO_CLOB() it is useless, manual cast could be necessary.
  * Replace IS JSON validation clause in CHECK constraints by
        (CASE WHEN $1::json IS NULL THEN true ELSE true END)
    When the code is invalid an error is fired.
  * DISTINCT and UNIQUE are synonym on Oracle.

Backward compatibility changes:

  - Force a column to be bigint if this is an identity column. Thanks
    to MigOps.com for the patch.
  - Fix EMPTY_LOB_NULL, enable/disable was inverted, keeping default
    to enabled. Take care that in old ora2pg.conf it is disabled so it
    will break backward compatibility with old configuration.
  - Replace NO_LOB_LOCATOR with USE_LOB_LOCATOR and NO_BLOB_EXPORT
    with ENABLE_BLOB_EXPORT to avoid confusion with double negative
    variable. Backward compatibility is preserved with a warning. 
  - SRID for SDO_GEOMETRY export is now taken from the value not forced
    from the metadata table. 

For a complete list of change see https://github.com/darold/ora2pg/blob/master/changelog

Ora2Pg v21.0 released!, October 12th 2020

This release fix several issues reported since last release and adds several new features and improvements.

New features, options and configuration directives in this release:

  * Add clause OVERRIDING SYSTEM VALUE to INSERT statements when the
    table has an IDENTITY column.
  * Considerably increase the speed to generate the report about the
    migration assessment, especially for database with huge number of
    objects.
  * Reduce time passed in the progress bar. Following the number of
    database objects we were spending too much time in refreshing the
    progress bar.
  * Add number of identity columns in migration assessment report.
  * Make assessment details report initially hidden using HTML5 tags
    
* Improve speed of BLOB/CLOB data export. Oracle recommends reading from and writing to a LOB in batches using a multiple of the LOB chunk size. This chunk size defaults to 8k (8192). Recent tests show that the best performances can be reach with higher value like 512K or 4Mb. * Add progress bar when --oracle_speed is use in single process mode. * Automatically activate USER_GRANTS when the connection user has no DBA privilege. A warning is displayed. * Complete port to Windows by using the Windows separator on stdout redirection into a file at ora2pg command line call and improve ora2pg_scanner port on Windows OS. * Add rewrite of MySQL JOIN with WHERE clause instead of ON. * Add MGDSYS (Oracle E-Business Suite) and APEX_040000 to the list of schemas excluded from the export. * Supply credentials interactively when a password is not defined in the configuration file. Need the installation of a new Perl module Term::ReadKey. * Add supports oracle connections "as sysdba" with username "/" and an empty password to connect to a local oracle instance. * Add translation of PRIVATE TEMPORARY TABLE from Oracle 18c into PostgreSQL basic temporary table, only the default behavior for on commit change.

New command line options:

  * Add new command line option to ora2pg_scanner: -b | --binpath DIR
    to set the full path to directory where the ora2pg binary stays.
    Might be useful only on Windows OS.
  * Add -r | --relative command line option and PSQL_RELATIVE_PATH
    configuration directive. By default Ora2Pg use \i psql command to
    execute generated SQL files if you want to use a relative path
    following the script execution file enabling this option will use
    \ir. See psql help for more information.

New configuration directives:

  * NO_VIEW_ORDERING:
      By default Ora2Pg try to order views to avoid error at import time
      with nested views. With a huge number of views this can take a very
      long time, you can bypass this ordering by enabling this directive.
  * NO_FUNCTION_METADATA
      Force Ora2Pg to not look for function declaration. Note that this
      will prevent Ora2Pg to rewrite function replacement call if needed.
      Do not enable it unless looking forward at function breaks other
      export.
  * LOB_CHUNK_SIZE
      See explanation in the new features and improvement list.
  * ALTERNATIVE_QUOTING_REGEXP
      To support the Alternative Quoting Mechanism ('Q' or 'q') for String
      Literals set the regexp with the text capture to use to extract the
      text part. For example with a variable declared as
            c_sample VARCHAR2(100 CHAR) := q'{This doesn't work.}';
      the regexp to use must be:
            ALTERNATIVE_QUOTING_REGEXP     q'{(.*)}'
      ora2pg will use the $$ delimiter, with the example the result will
      be:
            c_sample varchar(100) := $$This doesn't work.$$;
      The value of this configuration directive can be a list of regexp
      separated by a semi colon. The capture part (between parenthesis) is
      mandatory in each regexp if you want to restore the string constant.

Backward compatibility changes:

  - Default for NO_LOB_LOCATOR is now 1 to benefit from the LOB_CHUNK_SIZE
    performances gain.
  - Enable schema compilation (COMPILE_SCHEMA set to 1) by default to
    speed up DDL extraction.
  - Change the behavior of Ora2Pg with the parameters that follows a
    parameter with a default value. Ora2Pg used to change the order of the
    parameter's function to put all parameters with a default value at end
    of the list which need a function call rewrite. This have been abandoned
    now any parameter without default value after a parameter with a default
    value will be appended DEFAULT NULL.

For a complete list of change see https://github.com/darold/ora2pg/blob/master/changelog

Ora2Pg v20.0 released!, January 18th 2019

This release fix several issues reported during the last three months and adds several new features and improvement. The change of major version is related to backward compatibility break with the removed of most PG_SUPPORTS_* configuration directives and their replacement with the new PG_VERSION directive.

The release adds some new features and configuration directives:

  * Add PG_VERSION configuration directive to set the PostgreSQL major
    version number of the target database. Ex: 9.6 or 10. Default is
    current major version at time of a new release. This replace the
    old PG_SUPPORTS_* configuration directives.
  * Removed all PG_SUPPORTS_* configuration directives minus
    PG_SUPPORTS_SUBSTR that is related to Redshift engine.
  * Export of BFILE as bytea is now done through a PL/SQL function to
    extract the content of a BFILE and generate a bytea data suitable
    for insert or copy into PostgreSQL.
  * Foreign keys that reference a partitioned table are no more
    exported.
  * Show table name on Oracle side during export using at connection
    time: DBMS_APPLICATION_INFO.SET_ACTION(table_name);
  * When the date format is ISO and the value is a constant the call
    to to_date() is removed and only the constant is preserved. For
    example: to_date(' 2013-04-01 00:00:00','SYYYY-MM-DD HH24:MI:SS')
    is replaced by a simple call to: ' 2013-04-01 00:00:00'.
    This rewrite is limited to PARTITION export type when directive
    PG_SUPPORTS_PARTITION is enabled.
  * Add DATA_EXPORT_ORDER configuration directive. By default data
    export order will be done by sorting on table name. If you have
    huge tables at end of alphabetic order and are using multiprocess,
    it can be better to set the sort order on size so that multiple
    small tables can be processed before the largest tables finish.
    In this case set this directive to size. Possible values are name
    and size. Note that export type SHOW_TABLE and SHOW_COLUMN will
    use this sort order too, not only COPY or INSERT export type.
  * Add NO_BLOB_EXPORT configuration directive. Exporting BLOB could
    take time and you may want to export all data except the BLOB
    columns. In this case enable this directive and the BLOB columns
    will not be included into data export. The BLOB column must not
    have a NOT NULL constraint. Thanks to Ilya Vladimirovich for the
  * Add PREFIX_SUB_PARTITION to enable/disable sub-partitioning table
    prefixing in case of the partition names are a part of the sub-
    partition names.
  * Add special replacement for case of epoch syntax in Oracle:
        (sysdate - to_date('01-Jan-1970', 'dd-Mon-yyyy'))*24*60*60
    is replaced by the PostgreSQL equivalent:
        (extract(epoch from now()))

For a complete list of change see https://github.com/darold/ora2pg/blob/master/changelog

[...]