Skip to content
Merged
18 changes: 18 additions & 0 deletions src/changes/changes.xml
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,24 @@
</action>
<!-- dependabot:dep=actions/setup-java:new=5.6.0:pr=879 -->
<action dev="dependabot" type="update" due-to="Dependabot">Update github_actions dependency actions/setup-java from 5 to 5.6.0 (#879).</action>
<action dev="jeffjensen" type="fix" issue="492" system="github" due-to="jeffjensen">
Fix AbstractDatabaseConnection.getRowCount(String, String) ignoring DatabaseConfig.PROPERTY_ESCAPE_PATTERN, mirroring the escape-pattern handling createTable() already applies, so a configured escape pattern is no longer silently dropped from the generated "select count(*)" SQL.
</action>
<action dev="jeffjensen" type="fix" issue="628" system="github" due-to="jeffjensen">
Fix DatabaseTableMetaData.getPrimaryKeys() returning an empty array whenever a configured PROPERTY_PRIMARY_KEY_FILTER matches zero columns for a table, instead of falling back to the table's database-declared primary key. Note: this makes the fallback unconditional whenever a filter yields zero columns for a table, which now overrides a filter deliberately configured to declare "this table has no PK".
</action>
<action dev="jeffjensen" type="fix" issue="460" system="github" due-to="jeffjensen">
Fix DatabaseDataSet.initialize() querying the driver/database info for debug logging after already opening the table metadata ResultSet instead of before, which could silently invalidate that still-open ResultSet on JDBC drivers that allow only a single active cursor per connection (e.g. SQL Anywhere, some legacy Sybase drivers).
</action>
<action dev="jeffjensen" type="add" issue="171" system="github" due-to="jeffjensen">
Add opt-in DatabaseConfig.FEATURE_SORT_ALL_COLUMNS_WHEN_NO_PRIMARY_KEY, defaulting to off. When enabled, a table with no primary key is selected with an ORDER BY over all of its non-LOB columns instead of leaving row order database-defined (and thus nondeterministic); CLOB/BLOB columns are always excluded from that fallback sort since some databases (notably Oracle) reject LOB columns in ORDER BY.
</action>
<action dev="jeffjensen" type="fix" issue="542" system="github" due-to="jeffjensen">
Fix FlatDtdWriter.write() emitting an invalid, unparseable "&lt;!ELEMENT dataset (\n)&gt;" DTD content model for a zero-table dataset. It now emits "&lt;!ELEMENT dataset ANY&gt;" instead, which FlatDtdProducer already parses correctly and which, unlike EMPTY, still tolerates the whitespace a pretty-printed empty dataset file typically contains.
</action>
<action dev="jeffjensen" type="add" issue="533" system="github" due-to="jeffjensen">
Add MultiSchemaMySqlMetadataHandler, for a MySQL connection not restricted to a single schema (e.g. connecting as "root" specifically to work across several schemas at once with FEATURE_QUALIFIED_TABLE_NAMES enabled). MySqlMetadataHandler passes a null schema straight through as the JDBC catalog argument, which the JDBC spec defines as "search every catalog" but which MySQL Connector/J instead treats as "the connection's current catalog only" by default, silently hiding every other catalog's tables and producing NoSuchTableException for one of them. The new handler works around this by enumerating the connection's visible catalogs (skipping information_schema/mysql/performance_schema/sys) and unioning the per-catalog getTables()/getColumns()/getPrimaryKeys()/tableExists() results itself whenever no single schema is configured.
</action>
</release>
<release version="3.4.0" date="Jul 28, 2026" description="Test-suite hardening (un-skip and strengthen dozens of disabled/no-op tests); add CachingConnectionProvider and reduce DefaultPrepAndExpectedTestCase's per-test connection churn; pin identifier case-folding to Locale.ENGLISH for Turkish-locale correctness; and a broad set of correctness fixes across export formats (XML, YAML, CSV, XLS, Ant), TimestampDataType timezone handling, InsertOperation/TransactionOperation, and resource-leak cleanups">
<action dev="jeffjensen" type="fix" issue="797" system="github" due-to="jeffjensen">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@
public IDataSet createDataSet(String[] tableNames)
throws DataSetException, SQLException
{
logger.debug("createDataSet(tableNames={}) - start", tableNames);

Check warning on line 80 in src/main/java/org/dbunit/database/AbstractDatabaseConnection.java

View workflow job for this annotation

GitHub Actions / compile-and-unit-test

non-varargs call of varargs method with inexact argument type for last parameter;

Check warning on line 80 in src/main/java/org/dbunit/database/AbstractDatabaseConnection.java

View workflow job for this annotation

GitHub Actions / compile-and-unit-test

non-varargs call of varargs method with inexact argument type for last parameter;

return new FilteredDataSet(tableNames, createDataSet());
}
Expand Down Expand Up @@ -160,9 +160,12 @@
final StringBuilder sqlBuffer = new StringBuilder(128);
sqlBuffer.append("select count(*) from ");

String escapePattern = (String) getConfig()
.getProperty(DatabaseConfig.PROPERTY_ESCAPE_PATTERN);

// add table name and schema (schema only if available)
QualifiedTableName qualifiedTableName =
new QualifiedTableName(tableName, this.getSchema());
QualifiedTableName qualifiedTableName = new QualifiedTableName(
tableName, this.getSchema(), escapePattern);
String qualifiedName = qualifiedTableName.getQualifiedName();
sqlBuffer.append(qualifiedName);
if (whereClause != null)
Expand Down
11 changes: 8 additions & 3 deletions src/main/java/org/dbunit/database/AbstractResultSetTable.java
Original file line number Diff line number Diff line change
Expand Up @@ -132,11 +132,14 @@ protected AbstractResultSetTable(ITableMetaData metaData,

String escapePattern = (String) connection.getConfig()
.getProperty(DatabaseConfig.PROPERTY_ESCAPE_PATTERN);
boolean sortAllColumnsWhenNoPrimaryKey = connection.getConfig().getFeature(
DatabaseConfig.FEATURE_SORT_ALL_COLUMNS_WHEN_NO_PRIMARY_KEY);

try
{
String schema = connection.getSchema();
String selectStatement = getSelectStatement(schema, metaData, escapePattern);
String selectStatement = getSelectStatement(schema, metaData, escapePattern,
sortAllColumnsWhenNoPrimaryKey);

if (logger.isDebugEnabled())
{
Expand Down Expand Up @@ -181,10 +184,12 @@ protected Statement createStatement(IDatabaseConnection connection, int resultSe
return stmt;
}

static String getSelectStatement(String schema, ITableMetaData metaData, String escapePattern)
static String getSelectStatement(String schema, ITableMetaData metaData, String escapePattern,
boolean sortAllColumnsWhenNoPrimaryKey)
throws DataSetException
{
return DatabaseDataSet.getSelectStatement(schema, metaData, escapePattern);
return DatabaseDataSet.getSelectStatement(schema, metaData, escapePattern,
sortAllColumnsWhenNoPrimaryKey);
}

////////////////////////////////////////////////////////////////////////////
Expand Down
7 changes: 6 additions & 1 deletion src/main/java/org/dbunit/database/DatabaseConfig.java
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ public class DatabaseConfig
"http://www.dbunit.org/features/skipOracleRecycleBinTables";
public static final String FEATURE_ALLOW_EMPTY_FIELDS =
"http://www.dbunit.org/features/allowEmptyFields";
public static final String FEATURE_SORT_ALL_COLUMNS_WHEN_NO_PRIMARY_KEY =
"http://www.dbunit.org/features/sortAllColumnsWhenNoPrimaryKey";
Comment thread
coderabbitai[bot] marked this conversation as resolved.

/**
* A list of all properties as {@link ConfigProperty} objects.
Expand All @@ -111,6 +113,7 @@ public class DatabaseConfig
new ConfigProperty(FEATURE_SKIP_ORACLE_RECYCLEBIN_TABLES, Boolean.class, false),
new ConfigProperty(FEATURE_ALLOW_EMPTY_FIELDS, Boolean.class, false),
new ConfigProperty(PROPERTY_ALLOW_VERIFYTABLEDEFINITION_EXPECTEDTABLE_COUNT_MISMATCH, Boolean.class, false),
new ConfigProperty(FEATURE_SORT_ALL_COLUMNS_WHEN_NO_PRIMARY_KEY, Boolean.class, false),
};

/**
Expand All @@ -123,7 +126,8 @@ public class DatabaseConfig
FEATURE_BATCHED_STATEMENTS,
FEATURE_DATATYPE_WARNING,
FEATURE_SKIP_ORACLE_RECYCLEBIN_TABLES,
FEATURE_ALLOW_EMPTY_FIELDS
FEATURE_ALLOW_EMPTY_FIELDS,
FEATURE_SORT_ALL_COLUMNS_WHEN_NO_PRIMARY_KEY
};

private static final DefaultDataTypeFactory DEFAULT_DATA_TYPE_FACTORY =
Expand All @@ -150,6 +154,7 @@ public DatabaseConfig()
setFeature(FEATURE_CASE_SENSITIVE_TABLE_NAMES, false);
setFeature(FEATURE_DATATYPE_WARNING, true);
setFeature(FEATURE_ALLOW_EMPTY_FIELDS, false);
setFeature(FEATURE_SORT_ALL_COLUMNS_WHEN_NO_PRIMARY_KEY, false);

setProperty(PROPERTY_STATEMENT_FACTORY, PREPARED_STATEMENT_FACTORY);
setProperty(PROPERTY_RESULTSET_TABLE_FACTORY, RESULT_SET_TABLE_FACTORY);
Expand Down
59 changes: 54 additions & 5 deletions src/main/java/org/dbunit/database/DatabaseDataSet.java
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@
import java.sql.DatabaseMetaData;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Locale;
import org.dbunit.DatabaseUnitRuntimeException;
import org.dbunit.dataset.AbstractDataSet;
Expand All @@ -38,6 +40,9 @@
import org.dbunit.dataset.ITableMetaData;
import org.dbunit.dataset.NoSuchTableException;
import org.dbunit.dataset.OrderedTableNameMap;
import org.dbunit.dataset.datatype.BlobDataType;
import org.dbunit.dataset.datatype.ClobDataType;
import org.dbunit.dataset.datatype.DataType;
import org.dbunit.dataset.filter.ITableFilterSimple;
import org.dbunit.util.QualifiedTableName;
import org.dbunit.util.SQLHelper;
Expand Down Expand Up @@ -115,11 +120,19 @@ public DatabaseDataSet(IDatabaseConnection connection, boolean caseSensitiveTabl

static String getSelectStatement(String schema, ITableMetaData metaData, String escapePattern)
throws DataSetException
{
return getSelectStatement(schema, metaData, escapePattern, false);
}

static String getSelectStatement(String schema, ITableMetaData metaData, String escapePattern,
boolean sortAllColumnsWhenNoPrimaryKey)
throws DataSetException
{
if (logger.isDebugEnabled())
{
logger.debug("getSelectStatement(schema={}, metaData={}, escapePattern={}) - start",
schema, metaData, escapePattern);
logger.debug("getSelectStatement(schema={}, metaData={}, escapePattern={}, "
+ "sortAllColumnsWhenNoPrimaryKey={}) - start",
schema, metaData, escapePattern, sortAllColumnsWhenNoPrimaryKey);
}

Column[] columns = metaData.getColumns();
Expand All @@ -130,6 +143,12 @@ static String getSelectStatement(String schema, ITableMetaData metaData, String
"Cannot load data for " + metaData);
}

Column[] sortColumns = primaryKeys;
if (sortColumns.length == 0 && sortAllColumnsWhenNoPrimaryKey)
{
sortColumns = nonLobColumns(columns);
}

// select
final StringBuilder sqlBuffer = new StringBuilder(128);
sqlBuffer.append("select ");
Expand All @@ -150,7 +169,7 @@ static String getSelectStatement(String schema, ITableMetaData metaData, String
metaData.getTableName(), schema, escapePattern).getQualifiedName());

// order by
for (int i = 0; i < primaryKeys.length; i++)
for (int i = 0; i < sortColumns.length; i++)
{
if (i == 0)
{
Expand All @@ -160,13 +179,39 @@ static String getSelectStatement(String schema, ITableMetaData metaData, String
{
sqlBuffer.append(", ");
}
sqlBuffer.append(new QualifiedTableName(primaryKeys[i].getColumnName(), null, escapePattern).getQualifiedName());
sqlBuffer.append(new QualifiedTableName(sortColumns[i].getColumnName(), null, escapePattern).getQualifiedName());

}

return sqlBuffer.toString();
}

/**
* Filters out CLOB/BLOB columns, which many databases (notably Oracle) reject in an
* {@code ORDER BY} clause. Checks by type rather than identity against {@link DataType#CLOB}/
* {@link DataType#BLOB} so vendor-specific LOB data types are also excluded, e.g. a
* {@code OracleDataTypeFactory}-produced column typed {@code OracleClobDataType} or
* {@code OracleBlobDataType}, neither of which equals the generic singleton.
*
* @param columns The columns to filter.
* @return A new array containing every column from <code>columns</code> whose data type is
* not a {@link ClobDataType} or {@link BlobDataType}.
*/
private static Column[] nonLobColumns(Column[] columns)
{
List<Column> nonLobColumns = new ArrayList<Column>(columns.length);
for (int i = 0; i < columns.length; i++)
{
Column column = columns[i];
DataType dataType = column.getDataType();
if (!(dataType instanceof ClobDataType) && !(dataType instanceof BlobDataType))
{
nonLobColumns.add(column);
}
}
return nonLobColumns.toArray(new Column[0]);
}

/**
* Get all the table names form the database that are not system tables.
*/
Expand Down Expand Up @@ -204,11 +249,15 @@ private void initialize(String schema) throws DataSetException
String[] tableType = (String[])config.getProperty(DatabaseConfig.PROPERTY_TABLE_TYPE);
IMetadataHandler metadataHandler = (IMetadataHandler) config.getProperty(DatabaseConfig.PROPERTY_METADATA_HANDLER);

if(logger.isDebugEnabled())
{
logger.debug(SQLHelper.getDatabaseInfo(databaseMetaData));
}

ResultSet resultSet = metadataHandler.getTables(databaseMetaData, schema, tableType);

if(logger.isDebugEnabled())
{
logger.debug(SQLHelper.getDatabaseInfo(jdbcConnection.getMetaData()));
logger.debug("metadata resultset={}", resultSet);
}

Expand Down
4 changes: 4 additions & 0 deletions src/main/java/org/dbunit/database/DatabaseTableMetaData.java
Original file line number Diff line number Diff line change
Expand Up @@ -382,6 +382,10 @@ public Column[] getPrimaryKeys() throws DataSetException
if (primaryKeysFilter != null) {
_primaryKeys = Columns.getColumns(getTableName(), getColumns(),
primaryKeysFilter);
if (_primaryKeys.length == 0) {
String[] pkNames = getPrimaryKeyNames();
_primaryKeys = Columns.getColumns(pkNames, getColumns());
}
} else {
String[] pkNames = getPrimaryKeyNames();
_primaryKeys = Columns.getColumns(pkNames, getColumns());
Expand Down
19 changes: 15 additions & 4 deletions src/main/java/org/dbunit/dataset/xml/FlatDtdWriter.java
Original file line number Diff line number Diff line change
Expand Up @@ -69,12 +69,23 @@ public void write(IDataSet dataSet) throws DataSetException
String[] tableNames = dataSet.getTableNames();

// dataset element
printOut.print("<!ELEMENT dataset (\n");
for (int i = 0; i < tableNames.length; i++)
if (tableNames.length > 0)
{
printOut.print("<!ELEMENT dataset (\n");
for (int i = 0; i < tableNames.length; i++)
{
_contentModel.write(printOut, tableNames[i], i, tableNames.length);
}
printOut.print(")>\n");
}
else
{
_contentModel.write(printOut, tableNames[i], i, tableNames.length);
// An empty parenthesized content model, e.g. "(\n)", is not
// valid DTD syntax. ANY tolerates the whitespace that a
// pretty-printed empty "<dataset></dataset>" typically contains,
// unlike EMPTY which rejects any content at all.
printOut.print("<!ELEMENT dataset ANY>\n");
}
printOut.print(")>\n");
printOut.print("\n");

// tables
Expand Down
Loading
Loading