databases/cego: update 2.33.5 -> 2.33.6
lfcbase:
- Small correction in OutStream, include required by FreeBSD
( for definition of WEXISTATUS )
- Added method getErrorCode to OutStream class
- Cleanup of OutStream implementation ( removed some includes,
removed ifdef for MINGW)
cego:
- Small improvements for utility scripts cgmkdb, logManager and backupManager
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoSelect and friends for query cleanup. The cleanUp
method did not cleanup all field value references which might lead
to invalid memory access in case of repeated view calls ( Variable
pParentJoinBuf ).
Corresponding cleanUp method have been introduced or CegoExpr,
CegoTerm, CegoFactor and friends.
- Check052 has been introduced to verify the previous described behaviour
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoTableManager::insertDataTable, the newRBEntry method
call for the transactions manager also must be catched by the
exception. Otherwise, the index might be corrupted if any exception
occur ( e.g. out of temp space )
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: update 2.33.1 -> 2.33.3
lfcbase:
- Small fixes for Monitor and Pager class, adding missing ifdef for
curses option in Pager class, replaced CGNOCURSES with LFCNOCURSES
cego:
- Fix in CegoRecoveryManager::recoverTableSet, before calling
transactionRecovery, we have to set flag setIgnoreInvalid to true.
During transaction recovery phase, we must ignore invalid btrees/index
objects since these objects could be invalidated but Log data
integrity and consistency is ensured
- Added implizit tableset btree and index correction to recovery procedure
( CegoRecoveryManager::recoverTableSet )
- Small syntax for cgadm from "backup TS" to "backup tableset TS"
( conform to other tableset commands )
- Fix in CegoTableManager::deleteDataTableEntryAtomic /
updateDataTableSynced / beginTransaction / commitTransaction /
rollbackTransaction to avoid log entries for forced transactions
( introduction of doLog flag ).
- Fix in CegoTableManager:deleteDataTable to avoid object synchronisation
during recovery phase ( introduction of isSynced flag ), since
during recovery phase, still no objects are registered and there is
no need for synchronization
- Added further admin console messages for recovery procedure.
This enables the database admin to follow the recovery procedure on
admin console level
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: update 2.32.14 -> 2.33.0
lfcbase:
- Added CommandExecuter::executeStream method to catch stdout for
any forked process
- Added class OutStream to catch stdout from forked programs via popen
cego:
- Added database verify006 to check multidimensional btrees also in
combination of null value handling.
- Fix in CegoObjectCursor::getNext, double method call for unlockData removed
- Small grammar correction to treat distinct flag for select-star statement
- Fix in CegoBTreeCursor::compValue, null values have not been handled
correctly. For string values, string termination was not clean, which
also might lead to wrong cursor results
- Introduced archive manager, which can be triggered, to perform
an online backup. For now, online backups have to be triggered
externally.
The achive manager should make backup / restore handling more convenient
- Update to version 2.33.0 since the backup feature changes the product
specification.
- Integration of newly introduced lfcbase OutStream class to catch output
from backup manager. This allows to indicate a more detailed
output from backup manager on the admin console.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: update 2.32.11 -> 2.32.14
lfcbase:
- Small fix for Monitor::showFormBox, in case of menu elements,
delete key is disabled
cego:
- Correstion for distinct clause. With the current grammar implemention,
queries with nested selects and using the distinct operator might
fail. Cego.def grammar and CegoAction implementation has been changed
in way, that distinct option is just allowed at the beginning ( e.g.
select distinct count(a) from t1 and NOT select count(distinct a) from t1 )
This should simplify usage, since by using the distinct clause,
distinct tuples are retrieved either for plain queries or for
aggregation
Confusing sample not allowed anymore : select count(distinct a),
count(distinct b) from t1 ... now select distinct count(a), count(b)
from t1 )
- Fix in CegoBTreeValue::valueFromSchema added.
In case of multidimension btree objects, the insert of new values
might fail, if string bases attributes occur at the beginning
- Fix in CegoDistManager::registerObjects, registration of btree
objects was still missing. This might lead to wrong results for
drop if exists btree statements
- Added check to CegoXMLSpace::addArchLog to verify, that directory
for new archive location really exists
- Added archive log management to curses based CegoAdmMon administration
console
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: update 2.32.8 -> 2.32.11
databases/cegobridge: update 1.1.2 -> 1.1.3, fix build with clang 4.0
devel/lfcbase:
- Added Monitor class for ncurses based GUI utility implementation.
This class provides GUI layout and event handling to simply ncurses
based GUI tool implementaton.
- Added class Pager for curses based pager implementation
databases/cego:
- Include stdlib.h added in CegoNet ( still missing forLinux compiles )
- Added CegoFieldValue::getDim method required for arithmetic operations
on fixed values
- Fix in CegoTableManager::getPoolInfo, readdelay and writedelay was
not calculated correctly
- Fix in CegoDistManager::stopDistTableSet, removing of btree objects
from database object dictionary was still missing
- In CegoTableManager::dropTableSet now also counter objects are
dropped ( stored in the database xml file ).
- Added some sizing optimizations for CegoAdmMon forms
- Fix in CegoRecoveryManager::recoverCurrentTransactionLog, added
the force option to addCounter method, since counters already could
be synched to xml ( not completely transaction save )
- Online index build up tested and verified. First tests with heavy
insert operations ( using cgblow ) during index build went succesful.
- Improvements for CegoAdmMon role permission handling, added remove
permission option
- Preparation of CegoObjectCursor and CegoTableManager to support
online index rebuild with write access to the table.
For this, concurrent insert operations have to be performed with
append =true. The ObjectCursor locks the last page until the index
rebuild is finished ( method setLastPageSync )
In this way, the index can be build up in parallel, while further
tuples could be appended to the table. ( just supported for btree
index objects )
PR: 216621
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
Reported by: jbeich
databases/cego: update 2.32.7 -> 2.32.8
lfcbase:
- Changed methods for class Sleeper to static. Added Sleeper class
test to test suite
- Added include file limits.h in Chain.cc. For Linux compiles, this
include is required ( definition of INT_MAX and INT_MIN )
cego:
- Small improvements for CegoAdmMon curses based admin console
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoFileHandler::releaseFiles.
_tabSetId and _isReg vars have to be set to zero/false in any case.
Otherwise, it might happen that old entries are still visible and in case
of a tableset recreation obsolete file handle information is provided
- Method CegoTransactionManager::hasOpenTransaction added to check for open
transaction for a spedific table. This method is called now before a table
can be dropped to ensure there are no open transaction for this table.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Small performance improvement in CegoBufferPool::bufferFix
Instead of copying the buffer head data to a local variable via
memcpy, a pointer variable is used to access the data directly
Performance improvement: just a few percent ...
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Added another performance patch to CegoBTreeCursor.
The method compValue has been redesigned in a way, that comparison
is prepared at cursor creation time and so could be executed in
a optimized and faster way
This speeds up queries with joined tables, where the joined table
is traced many times ( e.g. select count(*) from t1, t2 where
t1.a = t2.a )
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fixes and improvements for table export added
In CegoXPorter::xmlExportable, the pOutStream pointer was deleted,
which lead to a core dump ( pointer is already deleted by Element )
Row counter handling and messages have been improved
- Change back to memove call in CegoBTreeNode:shiftEntries. Copying manually
and bytewise decreased performance about two times for building
up btree objects. This has been observed on native POSIX ( FreeBSD 11 )
but also on MSYS64 compiles
- Improvements for CegoBTreeCursor. With the new introduced methods
CegoBTreeNode::getMedPage and CegoBTreeNode::rightValue, the search
for the appropriate subnode can be made with logarithmic efforts
instead of linear search. To validate this improvement, the following
query has been setup for a random filled table t1
select * from t1 ta where exists ( select * from t1 tb where tb.b = ta.b );
Execution time with the improved btree cursor was two times faster than with
old implementation.
- More performance improvements made for btree creation
changed linear search efforts in CegoBTreeNode::getChildPage to logarithmic
search efforts ( introduced lb /rb variable )
- Change in CegoBTreeManager::allocPage. Since for cache based creation,
just a pageId must be allocated, the low level method
CegoFileHandler::allocatePage is used.This avoids an unnessary
bufferFix call for the new page
- Introduced appendFid variable for CegoFileHandler class. In case of
many managed datafiles, the last file with avaiable pages is used
for the allocatePage method.
- Performance comparison to MariaDB indicates now a better scaling
from 2 million rows and up ( by using cego btree cache )
- Improvement modifications for btree implementation increased build
up speed for just about 3 percent. It seems, the most time is used
by calling shiftEntries method, where memory is moved via memmove.
I observed, that 8k page size ( instead of 16k ) brought up some
performance improvement for btree creation of about 100% ( for
type string(60) with random values )
- More performance improvements in CegoBTreeManager and CegoBTreeNode,
overall impact still has to be analysed
- Fix in CegoTableManager added for alter table modify column,
columns with existing btree or key objects can not be modified
anymore
- Further improvement approach in CegoBTreeManager to increase btree
creation performance. BufferPage handles have been changed from
native variables to pointer handles. This should reduce page access
efforts in case of enabled btree cache
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoAction::insertStore and CegoAction::insertValueSpecStore;
for value array inserts via network, the ack for the protocol was
no satisfied ( e.g. insert into t1 values ( 1, 'XXX'),(2,'YYY'),...)
which leads to a hanging database client. Since class CegoQuey
was already prepared for bulk inserts, this was a simple fix,
just calling execQuery via insertStore parser method just one time.
- Code cleanup in CegoBTReeManager and CegoBTreeNode, use of static
method CegoDataPointer::getEncodingLength, (define of DPENCODINGLENGTH)
- Fix in CegoFileHandler::CleanDataFile,pageId was not calculated
correctly ( missing offset )
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoTranactionManager, replaced LONG_TYPE with PAGEID_TYPE,
since this is the correct type for page entries. This was detected
on MinGW64 compiles (seg fault occured), since the type size differs
on this platform.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fixes for CegoBTreeManager::deleteBTree ( changed int type to
PageIdType for page vars )
and CegoBTreeManager::insertBTree ( propValue was not setup correctly
for root node propagation in some cases, which might lead to corrupt
btree structure )
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
lfcbase:
- Added range check to Chain::toInteger method to catch overflow exception
cego:
- This version brings a complete redesign of low level page handling.
Instead of page references identified by fileId and pageid, a
database unique pageid is used now
This results in a complete reimplemtation of several low level
classes like CegoFileHandler, CegoBufferPool, Blob handling, etc.
Since pages are references by a single ( 64 bit ) id now, I expect
an increased performance behaviour over all database operations.
Most code modifications are done, code complies and basic
functionally works ( create tableset, create table, insert table )
- First performance analysis indicates a speedup of about 10% for
btree creation, so significant speedup for full table scans.
- All base checks passed, but there is still a page allocation leak
for table drops
- Functional tests with SysMT successful completed
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoFieldValue for fixed value handling.
In case of fixed value aggregations, the specified fixed dimension
should be kept. This has been done be analysing the value dimension
in the CegoFieldValue:add/sub/mu/div methods.
In CegoQueryHelper::prepareFieldValue, fixed values are checked now
for right dimension
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
The only reason to use post-stage is because the port needs to do
"things" at a later time, like some plist manipulation.
While there, fold post-install in do-install targets when they are
defined.
PR: 214780
Submitted by: mat
Exp-run by: antoine
Sponsored by: Absolight
- Fix in CegoSelect::getQueyId added, for the ordering clause asc
and desc ordering attributes were missing. This might lead to wrong
ordering results if query caching is used
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Changed CegoQueryHelper methods to static
- Fix for CegoBTreeManager and friends regarding small string values
( string(1) )
- Please note, that this fix invalidates btree format.
Btrees have to be recreated after this fix, databases need to
be exported and imported.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fixed a file handle leak for datafiles and logfiles ( CegoFileHandler,
CegoTableManager and CegoLogManager ) After tableset creation (
or start and stop ), file handles have not cleaned up properly.
This lead to file remove problems in case of a tableset drop.
This effect was detected by building and testing the MinGW64 port
Submitted by: lemke@lemke-it.com
- Fix in CegoClient for escape character handling in batch mode
- Fix in CegoClient for dumpfile generation. Default values handling corrected
- Join optimizer fix in CegoSelect, CegoAttrCond and CegoBtreeCursor.
join attributes are checked now in a more sophisticed way, which
leads to improved execution performace for advanced joins
PR: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoFunction::evalFieldValue, for functions date2str,
date2int, int2date, type casting to appropriate type has been added.
Without the cast, the functions might return wrong results
- Adaptions for CegoBlowTab to use CegoNet API
- Completion for blob methods in CegoNet
- Improved flow control in CegoClient ( based on ncurses )
- Optimization in CegoClient, just high level CegoNet API methods
are used now for client implementation. Missing methods have been
added to CegoNet API
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoNet::connect, the global vars __dateTimeFormat,
__currencySymbol and __decimalPoint are set up now to appropriate
defaults
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.30.21 -> 2.30.23
lfcbase:
- Improvements for BigDecimal class
All values are normalized now and stored now with non trailing zeros
cego:
- Fixes for cast operations regarding values for type fixed
- Fix in CegoClient, global vars __currencySymbol and __decimalPoint
still have not been set up correctly, done in the same way as for
CegoMain on server side
- Added verify005.sql to dbcheck suite to verify cast operations
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoProcedure::execute and CegoProcedure::getRetVal In the
castTo method, the type dimension was still not treated. This might
lead to wrong cast results especially for fixed values
- Fixed cgadm online help for import command, the nologging option
was not documented
- In CegoRecoveryManager, added tableset argument line argument for
calling the external log manager.
This makes it more easy in the log manager code, to implement
tableset dependent restore strategies
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.30.17 -> 2.30.19
lfcbase:
- Added File::rename method to be able to move files via file class
- Patch file File class, operator= and operator+= did not close the
file descriptor after copying / appending. This may cause a problem
for subsequent method calls on the instance. ( e.g. renaming or
removing )
This effect actually has just been observed for MinGW64 compiles
cego:
- Catched server termination in CegoTableManager::writeCheckpoint.
If log files could not be archived, the method does not recognized
server termination ( runstate = OFFLINE )
- Improvement added for log file shipping in CegoLogThreadPool::copyLog.
The archive log file is first written to a temporary file and if
finished, is moved than to its original file name. For the tape
management, this is an indication, that the file is completely
written and is ready for further processing
- The previous patch in lfcbase required a patch for
CegoLogThreadPool::copyLog. Since the archfile file was not closed
after trunc, the rename operation failed now. close method call was
added
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fixed for point-in-time recovery, Date format handling was not
done correectly
- Elimination of BUFFERPOOLHASHKEY. The bufferpool position is now
calculated based on the page offset of the corresponding datafile
( this information is provided by CegoFileHandler::getRegPageOffset )
This leads to a much better distribution and usage of the available
buffer pool pages
- Fix in CegoTableManager::createBTree, the creation of the btree object
must be done in a dedicated exception block, otherwise the btree
object is dropped, if it already exists.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.30.14 -> 2.30.15
lfcbase:
- Added statlock mutex in ThreadLock class for protecting statistic
counters. It seems, increment and decrement operators on atomic
values are no more thread safe ( Once it was, but I guess this is
a compiler issue )
Since the lock statistics can be disabled for production mode,
there is not performance impact caused by the additional mutex
calls
cego:
- Made __dateFormatList in CegoMain threadsafe
For the point-in-time recovery timestamp format, now the server
defined format is used ( variable CGPITFORMAT is obsolete )
- Added update sync mode. With this mode, the user can decide, if
an update or delete should be blocked until another concurrent
transaction is finished or return immediatly without updating
concurrent affected tuples
- Reactivation of rollback segment record locks in CegoLockHandler class.
In CegoObjectManager::insertData, in case of rollback segements
insert operations, the RB record lock is used instead of the
system record lock. This might avoid some deadlock situations.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Added alter table modify command just for default values.
alter table t1 modify column a default 42;
- Improvement for hash calculation in CegoBufferPool::calcSegement.
For a better page spreading, the segment calculation was changed
to a more linear approach
- Added more cast cases to CegoField::castTo, still not completed
- Cast handling in CegoFieldValue::castTo completed.
This allows an any to any value casting, except lob values and
some unsupported datetime casts ( e.g. bool to datetime really
makes no sense )
- Fix in CegoBufferPool::poolInfo, the spread rate calculation was
not done correctly
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Added cast improvements to CegoFieldValue::castTo method, which are
also checked now with check050
- Evaluation of locale to find out decimal point character. E.g.
for english locales this is ., for german this is ,. The character
is used for float value handling in CegoFieldValue methods.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fix in CegoRecoveryManager::recoverTableSet, the lsn was incremented
which lead to leaks in the transaction log. This could result in
an incomplete recovery procedure
- Improvements for tableset recovery, external log manage field is
checked in CegoAdminThread::medRecover before restore procedure is
started.
- Fix in CegoLogManager::logAction, if online redo log is full, the
current log entry has to be written anyway, otherwise a checkpoint
is written and subsequent logAction call in CegoTableManager::logIt
might lead to duplicate entries in case of a crash recovery
- Fixes for query and table cache added (addEntry method was still not clean)
- Added tableset runstate CHECKPOINT to be set during a tableset
checkpoint is performed.
Since this is a critical section in terms of data consistency,
no system crash should occur while writing a checkpoint.
If so, the tableset now is marked with status CHECKPOINT and cannot be
used anymore ( tableset recovery from backup required )
- Patch in CegoDistManager::startTableSet, before starting redo log
file recovery, the tableset state has to be set to RECOVERY
Otherwise, occupied redo logs are not archived by the log manager.
This problem might occur in case of a crash recovery with archive
mode set to ON
- Patch in CegoQueryCache and CegoTableCache ( for each method addEntry )
The usedSize calculation was not done correctly for replaces entries
- Added table and cache used size information to show tableset admin command
This indicates the amount of memory, which is already allocated
for cached data
- Added synrchonization for query and table cache handling.
The lack of synchronization might lead to core dumps, if a cache
entry is freed but also be used by a query
So before cache cleanup, it has to be checked,
if the corresponding cache entry is still in use
If so, it has to be wait, until the cache is clean. For both
caches ( query and table), the similar logic is used
- Added improvement for query cache handling
If a stored function, which are used in the cached query, was
modified, the cache entry will be invalidated
For this, the CegoSelect::getTableList method was changed to
CegoSelect::getObjectList and also the used functions are evaluated
and added to this list.
- Adapted sysmtexp and sysmtimp X-port utilities for current schema definition
- Cache cleanup in CegoQuery class is performed now AFTER the
modifying action.Otherwise, it may appear, that a cache entry is
made after cache cleanup but also before the modifying action.
This may lead to wrong results retrieved from obsolete query cache
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Fixed memory leak in CegoSelect::nextTuple.
Since the querycache is allocated by the CegoQueryCache instance, the
pCacheList variable has to be freed after calling the addEntry method.
Actually, the logic is not very smart, since the cache element is build up
two times...
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
The following query can cause a core dump
create table t1(a string(100));
insert into t1 values ( 'Hugo Habicht');
insert into t1 values ( 'Kim Kaiser');
select substr(a, 1, getpos(a, ' ')) from t1;
For return value length evaluation, the method CegoFieldValue::asInteger
is called on a null value which causes the dump. A fix has been
added, so the method returns a value of 0 in case of null values.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
devel/lfcxml: 1.2.3 -> 1.2.4
databases/cego: 2.30.3 -> 2.30.5
lfcbase:
- code format cleanup
lfcxml:
- code format cleanup
cego:
- Code optimizations tested. The performance improvements on
subselects are achieved as expected.
- Code optimization for join buffer handling. Now a double pointer
list is used to just keep references on CegoField lists. This should
avoid expensive memory copies in case of creating parent join buffers
for sub selects.
- code format cleanup
- Fix in CegoAdmMon, CGNOMONITOR definition was fixed, now defined
via configure
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
Changes:
- Fix in CegoBufferPool::bufferFix. Small leak, since last page for each
pool segment never could be used ( hashId calculation )
- Cosmetic syntax change for set permission and remove permission admin
command
- Small information fix for query cache information in show tableset
admin command
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.29.3 -> 2.30.2
Changes:
- Fix in CegoAction and CegoDbHandler to handle query abort requests correctly
( initiated by cgadm via abort db thread command )
- Fix in CegoDistCursor for query plan calculation ( join objects )
- Fix in CegoAdminThreadPool and CegoLogThreadPool for thread load
calculation
- Major change for tablecache and querycache management,
for both dedicated tableset entries are managed. Cache enabling,
cache size and num entries can be controlled now on tableset level.
- Fixes for date format handling. The format string whihc is specified
for the server is also used now for client formatting. This results
in some network protocol fixes for serial but also xml network
protocol.
- Changed default protocol from xml to serial. Date format handling
completely implemented
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.29.1 -> 2.29.3
Changes to lfcbase:
- Fix in Chain for truncleft, truncRight and cutTrailing. In case
of single character strings, the methods might return wrong results.
- Started with implementation of TextIndex class ( utility class
for full text index creation )
Changes to cego:
- Fixes for table and query cache handling
- Fix in CegoDistManager::dropDistObject, table and query cache
have been checked for cleanup, if a table is dropped
- Fix in CegoQuery.cc, added table and query cache cleanup for alter
and rename queries
- Fix in CegoClient to parse multi line statements correctly. Further
multi line merge token ( \ ) has been added to merge long string or
clob values. Now the following statements are possible:
insert into t1 values ( 1, 'this is\
a multi line \
message \
with four lines');
- Added include of string.h in CegoAction.c ( compile error occured
on Suse Linux )
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.28.3 -> 2.29.1
databases/cegobridge: 1.1.1 -> 1.1.2
Please note: Updating cego databases needs an export/import
cycle for 2.28 -> 2.29
lfcbase:
- Fix in Datetime.cc, added include config.h, otherwiese the local
implementation of strptime ist used. This should just be done, if
the system API does not provide this call ( e.g. MINGW ) .
- Memory leak fixed in File::readLine
cego:
- Started with clob datatype implementation
- Fix in CegoSelect constructor, _cacheEnabled var was not initialized
in the decoding constructor, which might lead to core dumps in case
of recovery of complex with enabled querycache Fix in CegoAdminThread
constructor, disableAuth method for TableManger called, otherwise
tableset recovery might fail with auth error
- Required network protocol expansions done for clob handling
(putclob, getclob, delclob)
- More work on clob handling. A fix was done for update operations
on clob values in CegoTableManager::updateTuple method. Before
updating the tuple, all clob references must be cloned, otherwise
in case of untouched clobs, references are invalid. This approach
is brute force, better would be, to stay on the untouched clob
references. But in terms of transaction and recovery handling,
this is not trivial
- Very long strings are handled now in CegoAction and CegoClient
to also parse large clob values
- Modification in cgclt for raw mode, Now just plain data is listed,
with the command line option --raw=X, a separator token can be setup
as a column separator
cegobridge:
- fix build problem
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
devel/lfcxml: 1.1.8 -> 1.2.2
databases/cego: 2.27.2 -> 2.28.3
databases/p5-DBD-cego: bump portrevision
databases/cegobridge: bump portrevision
IMPORTANT: Upgrades of cego 2.27.x to 2.28.x databases has to be done by
export and import.
lfcbase:
- Adaptions made for build with msys2 / mingw64. Many windows
specific stuff could be eliminated, since mingw64 provides better
POSIX support ( e.g. pthreads, timer functions .. )
lfcxml:
- Fix in configure.ac to find lfcbase and for FreeBSD
cego:
- Fix in CegoSelect::CegoSelect, the pCacheList variable was not
initialized to 0 for all constructors. In case of system table
selects, the variable might be undefined which might lead to
segmetation faults.
- Changed synchronization level for btree creation. So the base
table can also be accessed while btree is created ( During creation
btree status is invalid )
This also allows to perform on online btree reorganization, since
a new btree can be build up in parallel
- Fix in CegoSystemObject::createSystemSpace, id values for bustat
table must start with id 1 instead of id 0, otherwise, bustat
information retrievel fails or may result in a core dump
( select * from $bustat )
- Added performance fix for CegoQueryHelper::decodeFVL, since the
tuple context information is already from the method called, the
tid, tastep and ts parameters have been remove from this method.
This saves some decoding time
- Please note, that version 2.28.x makes datafiles from older versions
incompatible.
An upgrade requires a xml tableset export of existing data and a
restore after succesful upgrade
- 2.28.x also contains many code cleanup fixes and some fixes for
btree handling.
- Fix in CegoTableManager::createBTree, in case of an exception
during build up, the allocated have to be cleanup up. For this the
a rollback method was introduced in class CegoBTreeManager.
Fix in CegoTableCursor, invalid btree objects have not been ignored
- Many adapations to make cego full 64bit ready also in combination
with MSYS2 / MINGW64. This resulted in change of many variables
from type long to type unsigned long long (type long is just 32 bit
for Windows ) With an appropriate MSYS2 / MINGW64 installation,
cego compiles proper and runs stable now.
Many thanks to the MSYS/MINGW developers for their amazing work !
- Several changes from type int to type unsigned long long
- Fix in CegoTableManager::deleteDataTable ( renamed to
deleteDataTableEntry )
Since we allow now parallel updates, the method must check if the
target tuple has already been touched by another transaction.
Otherwise, invalid double entries could occur in case of concurrent
updates on the same tuple.
If a concurrent transaction is detected, the method returns false.
This return value is used by the updateTuple method to decide,
if the subsequent insert operation must be performed.
- Since it is not enough jut checking the tuple header info for
update deletes, a record locking has to be added to avoid duplicates.
In this sense, record locking for setTupleInfo method call ( used
in CegoTableManager and CegoTransactionManager ) was added. For
this, the lockRecord and unlockRecord method have been reactivated
in CegoLockManager class.
This lock avoids, that in case of concurrent tuple updates duplicate
tuples are produced by an update.
- Fix in configure.ac to find lfcbase and lfcxml and for FreeBSD
Submitted by: lemke@lemke-it.com
- Fix for system tables in CegoAttrDesc::evalReferences, for system
tables the evalTableReferences method has to be called, otherwise
attributes for system tables can not be accessed
- Add dbCheck for Fetch & Cache ( check047 ), all base checks are
run now with querycache ON and OFF
- Fix in CegoSelect::nextTuple, the cache schema was not set up
properly, if nextTuple was just called with empty jfl. This happens
in case of procedure cursors, ( CegoProcFetch ), where the field
value list is for all method calls empty
- Fix for CegoExpr::getAggregationList(), missing break operations
in switch block. This bug might lead to error messages regarding
aggregation queries like
CGCLT > select sum(a) + 0 from t1;
Error : Invalid expression sum(t1.a) + 0 for aggregation
- Modification for admin command set tsroot. Also the base path for
all defined datafiles is set now to new tsroot
- Fix in CegoBTreeCursor and CegoBTreeManager. Several conditons
have been changed from fileId && pageId to fileId || pageId
This bug may results in invalid btree objects in case of btree
spawn over several datafiles ( the pageId might be 0 in the next
file )
- Fix in CegoGroupSpace::insertTuple, the grouping tuple has to be
searched using getTableAlias instead of getTableName, otherwise the
following query returns wrong results
select count(*), a from t1 group by a; --works
select count(*), a from t1 tx group by a; -- broken
A check for this has been added in dbcheck suite
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
- Introduced BTree Cache to speed up the creating of large btree
indizes. A BTree Cache test done with 100 million tuples went
from 20h to 2.6h to create the index.
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
databases/cego: 2.26.14 -> 2.26.16
lfcbase:
- Improvement for ListT template class. The end of the list is now
stored int _listEnd variable. This improves Insert performance,
since the end of the list can immediately accesed instead of tracking
through the whole list until the end is reached.
- Patch in ThreadLock, the mutex for lock statistics has been
removed, since there is no need for synchronization. Also errors
have not been checked for the pthread_rwlock_wrlock, which might
cause some deadlocks.
cego:
- Ugly bug fixed in CegoObjectManager::releaseDataPtrUnlocked.
The logical operator && must be replaced by ||, otherwise fix leak arises
- Fix in CegoSelect::nextTuple for query cache to avoid, that queries
including system tables are cached.
- Error message fixes in CegoXPorter
- Introduced bufferpool distribution rate information in show pool output.
This indicates, if the buffer pool is used in a balanced way.
The behaviour can be changed with the BUFFERPOOLHASHKEY parameter.
The list pool command now receives the pool information in several
chunk messages. This allows to print also very large buffer pool
configurations
- Introduced BUFFERPOOLHASHKEY as a database tuning parameter. Based
on this parameter, the corresponding bufferpool slot for a fileId,
pageId tuple is calculated. Depending on the configured datafile
size, it might be useful to adjust this parameter
- Cego Tera Test started. Approach is to load about 1TB of random
data through the whole stack ( via cgblow ). Actually we are running
with 19 parallel processes loading concurrently 4 different tables.
Estimated date to achieve TB-Size is in about one month.
- Changed command line argument --nolockstat to --lockstat. So the
default value is lockstat disabled, since this has impact to the
database performance
- Modification of hash function in CegoBufferPool to achieve a
better distribution for large buffer pool configurations
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
devel/lfcxml: 1.1.7 -> 1.1.8
databases/cego: 2.26.13 -> 2.26.14
lfcbase:
- Added kill method to class Process, added SigTest check program to
basecheck suite
- Fix in BigDecimal::operator < and > .. comparison was not done correctly.
- Added automake check suite in baseecheck directory ( derived from samples )
lfcxml:
- Added automake check suite to package
- Small fix for XMLSuite constructor for pInStream initialization
cego:
- Added encoding implementation for CegoCaseCond to allow case
conditions also for update operations like
update erp_disk
set location = case when substr(wwn,20,1) = '1' then 'MD' else 'GV' end;
- Cosmetic changes in CegoDefs, increased FILEMNG_MAXDATAFILE to 5000
Submitted by: Bjoern Lemke <lemke@lemke-it.com>
