What do you want to do when repserver error # 5185 (row count mismatch) occurs for this particular database connection? [NOTE: ignore the 5189 in this case and pay attention to the 5185]

Typical options are ignore the error, stop_replication (ie, shutdown DSI), generate a warning message but continue processing, log the transaction to the RSSD and skip it, retry then log (retry_log) to RSSD and skip, retry and stop(retry_stop) if still not successful.

Assuming you want to ignore the error (ie, continue processing without stopping the DSI) then you need to modify the repserver errorclass that's bound to your DSI connection.

To find the name of the repserver errorclass (from within the RSSD): rs_helpdb <ds>,<db> (look under the repserver_errorclass column)

Next you need to find out if the repserver errorclass has a parent/controlling repserver (PRS) assigned to it (from within RSSD): rs_helpclass <repserver_errorclass_name>

If the repserver errorclass doesn't have a parent repserver (eg, undefined PRS) then you can assign the current repserver as the parent (from within the RS): create replication server error class <repserver_errorclass_name>

You now need to assign the desired action for error #5185 (from within the parent/controlling RS), using ignore as an example: assign action ignore for <repserver_errorclass_name> to 5185

See the Replication Server Reference Manual for details on the various commands mentioned above.

Keep in mind that by assigning this action (eg, ignore 5185) to a repserver error class, the same action will occur for all other database connections that are bound to this same repserver error class.

If you want to assign a specific action for 5185 to just the one database then you'll really want to create a new repserver error class from scratch ... relatively easy to do but requires a bit more understanding of how classes are managed by repservers, how said classes can be inherited by downstream repservers, and making sure the new repserver error class has a complete set of actions for all possible errors.

Hi Mark.

Plese see the example:

In the error log of Replication appears

T. 2014/01/30 18:08:11. (45): Last command(s) to 'BICCONTINGENCIA.consulta':

T. 2014/01/30 18:08:11. (45): 'begin transaction  [0a] delete from dbo.temp_01 where rut='1-9' and nom='JUAN' [0a] delete from dbo.temp_01 where rut='1-9' and nom='PABLO'

I. 2014/01/30 18:08:11. A grouped transaction of 2 individual transactions has failed in database 'BICCONTINGENCIA.consulta'. Each transaction in the group will be executed individually.

H. 2014/01/30 18:08:12. THREAD FATAL ERROR #5189 DSI EXEC(104(1) BICCONTINGENCIA.consulta) - /dsiutil.c(819)

        The DSI thread for the 'BICCONTINGENCIA.consulta' database is being shut down. DSI received Replication Server error #5185 which is mapped to STOP_REPLICATION or RETRY_STOP_REPLICATION. See the Replication Server log for more information.

T. 2014/01/30 18:08:12. (45): Last command(s) to 'BICCONTINGENCIA.consulta':

T. 2014/01/30 18:08:12. (45): 'begin transaction  [0a] delete from dbo.temp_01 where rut='1-9' and nom='JUAN' [0a] delete from dbo.temp_01 where rut='1-9' and nom='PABLO' '

W. 2014/01/30 18:08:12. WARNING #5185 DSI EXEC(104(1) BICCONTINGENCIA.consulta) - /dsiexec.c(13619)

        Row count mismatch for the command executed on 'BICCONTINGENCIA.consulta'. The command impacted 2 rows but it should impact 1 rows.

I. 2014/01/30 18:08:12. The error was caused by output command #2 of the failed transaction on table 'dbo.temp_01'.

I. 2014/01/30 18:08:12. The DSI thread for database 'BICCONTINGENCIA.consulta' is shutdown.

In the replication server  i execute:

1> admin who_is_down

2> go

Spid Name       State                Info                                    

---- ---------- -------------------- ----------------------------------------

      DSI EXEC   Suspended            104(1) BICCONTINGENCIA.consulta         

      DSI        Suspended            104 BICCONTINGENCIA.consulta     

For those errors always we execute the command:

resume connection to BICCONTINGENCIA.consulta skip tran

Now in the error log of Replication appears:

I. 2014/01/30 18:14:49. The DSI thread for database 'BICCONTINGENCIA.consulta' is started.

I. 2014/01/30 18:14:50. DUMP OF 1 SKIPPED TRANSACTIONS FOR DATABASE 'BICCONTINGENCIA.consulta'

I. 2014/01/30 18:14:50. begin transaction 

I. 2014/01/30 18:14:50. delete from dbo.temp_01 where rut='1-9' and nom='JUAN'

I. 2014/01/30 18:14:50. delete from dbo.temp_01 where rut='1-9' and nom='PABLO'

I. 2014/01/30 18:14:50. execute rs_update_lastcommit @origin = 103, @origin_qid = 0x00000000004033df00080198000b0008019800080000a2c3012df3f80000000000000001, @secondary_qid = 0x000000000000000000000000000000000000000000000000000000000000000000000000, @origin_time = '20140130 18:19:22:640', @conn_id = 0

I. 2014/01/30 18:14:50. if @@error <> 0 rollback transaction

I. 2014/01/30 18:14:50.  commit transaction

I. 2014/01/30 18:14:50. END DUMP OF 1 SKIPPED TRANSACTIONS FOR DATABASE 'BICCONTINGENCIA.consulta'

I. 2014/01/30 18:14:50. The first 1 transaction(s) for database 'BICCONTINGENCIA.consulta' have been logged into the exceptions log and skipped. They have also been logged either in repserver log, or the location specified by sysadmin dump_file command.

The question is:

what can i do for avoid lose the 2 transactions that appear in the log?

What should Ido torecover the transactions?

Thanks,

Rene

Well, you can cut-n-paste the transactions from the errorlog and into the RDB.  'course the problem with doing this is that you're now applying those transactions out of order, which in turn could leave your RDB out of sync with the PDB.

So, the answer to your question (how to recover the txns) is to make sure you don't skip them in the first place. [NOTE: It's very rare that I recommend anyone skip a tran ... and then only if I know exactly what's in the transaction and I've verified I can live without said transaction.]

In this case the DSI went down due to a rowcount mismatch.  You'll have to figure out if this rowcount mismatch is 'ok' in your environment ... and that's really going to depend specifically on your environment and the transaction that triggered the 5185.

So, how to keep from skipping the tran?

1 - when the DSI goes down run 'sysadmin log_first_tran'; this will dump the offending transaction to the RSSD (this will not delete the transaction from the DSI queue); you can then go view the transaction in the RSSD (rs_helpexception); from here you could a) manually apply the transaction against the RDB and then resume/skip the connection or b) decide the transaction is not needed and can be ignored so resume/skip (though if you're doing this then you've got a problem with your replication setup => remember, I rarely suggest just skipping a txn), or c) you could make the necessary changes to the RDB to insure the transaction won't trigger a rowcount mismatch and then resume the connection

2 - modify the repserver error class to take a different action when a 5185 occurs; in your system the current action is to stop the DSI; if you review the transaction and decide the rowcount mismatch is not an issue then you can modify the repserver error class to ignore 5185s and then resume the connection [see my previous posts about details on configuring the repserver error class]

If you decide you don't want to ignore the 5185, ie, you need to know about rowcount mismatches, then you're going to need to research why the rowcount mismatch is occurring and fix it ... and troubleshooting a rowcount mismatch is going to require a good bit of intimate detail about your PDB, RDB and repserver configurations.

You really have no way to control how many transactions are skipped when a 'resume connection to <dataserver>.<database> skip transaction' is executed since transactions are grouped for performance purposes.

However the  #5185 replication server warning you have in your replication server log is an after the fact message telling you the following:

This message appears if the affected number of rows is

different from the expected number of rows, after a command

that is not part of SQL Statement Replication, or a

stored procedure, or a row change with autocorrection

enabled is sent to the data server.

Which means the transaction was applied to the target and this was the result.

When doing a skip transaction you are just  seeing the results of  the transaction in the log that was affected and can go investigate the target table to see if this is a problem and you must investigate why it is happening?  You will also need to determine if it is necessary to resynch the primary and replicate tables.

What you want to investigate is why you are getting the #5185 warning in the first place?  What this indicates is either

1) The table is not in first normal form and does not have a unique index that can be used in the replication definition primary key to identify a single row in the target database.

2) If the table cannot be put into first normal form then the primary keys in the  replication definition do not contain the necessary columns to uniquely identify a row in the target database ?

If you cannot satisfy either of those conditions you will want to investigate if transactions applied to this table can be done using stored proc replication or use sql statment replication for this particular table.

If you are getting these #5185 messages during large batch operations updating or deleting specific tables then it might be best to use sql statement replication?

You really have no way to control how many transactions are skipped when a 'resume connection to <dataserver>.<database> skip transaction' is executed since transactions are grouped for performance purposes.

Good point.

At a previous client the automated/scripted solution to the DSI going down (due to a duplicate key issue) was to resume the connection by skipping the transaction.  The first time I saw this 'magic' in motion I pulled up the skipped exception to show to the local DBA ... 81 commands in the transaction, testing showed only 2 of the commands responsible for duplicate key issues, but all 81 commands thrown away. DBA still couldn't understand why his PDB and RDB were always out of sync   Eventually tracked the 'out of sync' issue to a) repeated executions (every 2 weeks) of the switch active command before replication was quiesced and b) wholesale skipping of transactions when duplicate keys were encountered by the DSI.

Hi Mark,

Thanks for your help.

I had an event in the errorlog of Replicator and i corrected following your recommendation (Item 1)

- I ran "sysadmin log_first_tran"

- I saw  the error detail in RSSD

- I modified the data in the RDB

-Finally i executed "resume connection" without skip tran

Best Regards,

Rene

HI Stephen,

I was able to complete replication configuration with manual materialization , however I have doubt on Failover steps and Installation guide provided by SAP doesn't cover this area and Sybase documentation as well. I can say Sybase documentation is pointing no where ......

I must admit confused which document to follow on Sybase replication

however below is out put of SAP_status path , it gives me feeling replication is working fine ,

1> sap_status path
2> go
1> sap_status path
Path                  Name          Value                    Info

--------------------- ------------- ------------------------ ----------------------------------------------------------

                      Start Time    2014-02-03 11:29:45.129  Time command started executing.

                      Elapsed Time  00:00:04                Command execution time.

sybpr1                Hostname      sap-dr-pr1              Logical host name.

sybpr1                HADR Status  Primary : Active        Identify the primary and standby sites.

sybdr2                Hostname      SAP-DR-RP                Logical host name.

sybdr2                HADR Status  Standby : Inactive      Identify the primary and standby sites.

sybdr2.sybpr1.TSN    State        Suspended                Path is suspended. Transactions are not being replicated.

sybdr2.sybpr1.TSN    Latency      Unknown                  No latency information for database 'TSN'.

sybdr2.sybpr1.TSN    Commit Time  Unknown                  No last commit time for the database 'TSN'.

sybdr2.sybpr1.master  State        Suspended                Path is suspended. Transactions are not being replicated.

sybdr2.sybpr1.master  Latency      Unknown                  No latency information for database 'master'.

sybdr2.sybpr1.master  Commit Time  Unknown                  No last commit time for the database 'master'.

sybpr1.sybdr2.TSN    State        Active                  Path is active and replication can occur.

sybpr1.sybdr2.TSN    Latency      Unknown                  No latency information for database 'TSN'.

sybpr1.sybdr2.TSN    Commit Time  2014-02-03 11:29:33.220  Time last commit replicated

sybpr1.sybdr2.master  State        Active                  Path is active and replication can occur.

sybpr1.sybdr2.master  Latency      Unknown                  No latency information for database 'master'.

sybpr1.sybdr2.master  Commit Time  2014-02-02 10:54:11.034  Time last commit replicated

**************

now when I wish to do switch over I have installed ASCS instance and during PAS installation it stop at start instance phase with saplikey error.... and when I had check R3trans below is output

PS C:\Users\tsnadm> R3trans.exe -x
This is E:\usr\sap\TSN\SYS\exe\uc\NTAMD64\R3trans.exe version 6.24 (release 740 - 19.03.13 - 20:13:03 ).
unicode enabled version
sizeof(Selection_t) = 272
2EETW169 no connect possible: "DBMS = SYBASE --- "
E:\usr\sap\TSN\SYS\exe\uc\NTAMD64\R3trans.exe finished (0012).
PS C:\Users\tsnadm> R3trans.exe -x
This is E:\usr\sap\TSN\SYS\exe\uc\NTAMD64\R3trans.exe version 6.24 (release 740 - 19.03.13 - 20:13:03 ).
unicode enabled version
sizeof(Selection_t) = 272
2EETW169 no connect possible: "DBMS = SYBASE --- "
E:\usr\sap\TSN\SYS\exe\uc\NTAMD64\R3trans.exe finished (0012).
PS C:\Users\tsnadm>

Attached Trans.log

further when I try to bring database TSN online  it gives error as below

2> go
1> online database TSN
Unable to obtain exclusive access to database 'TSN'.

Appreciate your kind help ...... I have raised SAP Incident as well but yet to hear from them

Regards
Mohammad

Hi Mohammad,

This topic probably deserves a different thread, now that you have replication configured.

And sorry about the documentation confusion - we hope to improve it.

For your current issues, I am not intimately familiar with the tools you are using, but I understand the R3trans program is used for copying data between systems.  But if you have already materialized the two systems, no further copy should be required.

The error from the log file indicates the utility is attempting to connect to the standby database.  Which we would not want and do not allow by default - since all activity should be applied to the primary and replication copies it to the standby automatically.

From your log file: 

4 ETW000 [ dblink ,00000] ***LOG BY0=>[ASE Error SQL9665]Adaptive Server is running in 'Standby' mode. The user does not have 'allow hadr login' privilege.

Regards,
Stephen

Dear Sybase Experts...

I was able to configure replication environment below is out put of SAP_status path , it gives me feeling replication is working fine ,

1> sap_status path
2> go
1> sap_status path
Path                  Name          Value                    Info

--------------------- ------------- ------------------------ ----------------------------------------------------------

                      Start Time    2014-02-03 11:29:45.129  Time command started executing.

                      Elapsed Time  00:00:04                Command execution time.

sybpr1                Hostname      sap-dr-pr1              Logical host name.

sybpr1                HADR Status  Primary : Active        Identify the primary and standby sites.

sybdr2                Hostname      SAP-DR-RP                Logical host name.

sybdr2                HADR Status  Standby : Inactive      Identify the primary and standby sites.

sybdr2.sybpr1.TSN    State        Suspended                Path is suspended. Transactions are not being replicated.

sybdr2.sybpr1.TSN    Latency      Unknown                  No latency information for database 'TSN'.

sybdr2.sybpr1.TSN    Commit Time  Unknown                  No last commit time for the database 'TSN'.

sybdr2.sybpr1.master  State        Suspended                Path is suspended. Transactions are not being replicated.

sybdr2.sybpr1.master  Latency      Unknown                  No latency information for database 'master'.

sybdr2.sybpr1.master  Commit Time  Unknown                  No last commit time for the database 'master'.

sybpr1.sybdr2.TSN    State        Active                  Path is active and replication can occur.

sybpr1.sybdr2.TSN    Latency      Unknown                  No latency information for database 'TSN'.

sybpr1.sybdr2.TSN    Commit Time  2014-02-03 11:29:33.220  Time last commit replicated

sybpr1.sybdr2.master  State        Active                  Path is active and replication can occur.

sybpr1.sybdr2.master  Latency      Unknown                  No latency information for database 'master'.

sybpr1.sybdr2.master  Commit Time  2014-02-02 10:54:11.034  Time last commit replicated

**************

I have no idea how to do switch over as matter of Fact SAP guide or any docuement doesn't mention this anywhere ...

I have given a try by installing ASCS instance on secondary node  and  during PAS installation it stop at start instance phase with saplikey error.... and when I had check R3trans below is output

PS C:\Users\tsnadm> R3trans.exe -x
This is E:\usr\sap\TSN\SYS\exe\uc\NTAMD64\R3trans.exe version 6.24 (release 740 - 19.03.13 - 20:13:03 ).
unicode enabled version
sizeof(Selection_t) = 272
2EETW169 no connect possible: "DBMS = SYBASE --- "
E:\usr\sap\TSN\SYS\exe\uc\NTAMD64\R3trans.exe finished (0012).
PS C:\Users\tsnadm> R3trans.exe -x
This is E:\usr\sap\TSN\SYS\exe\uc\NTAMD64\R3trans.exe version 6.24 (release 740 - 19.03.13 - 20:13:03 ).
unicode enabled version
sizeof(Selection_t) = 272
2EETW169 no connect possible: "DBMS = SYBASE --- "
E:\usr\sap\TSN\SYS\exe\uc\NTAMD64\R3trans.exe finished (0012).
PS C:\Users\tsnadm>

Attached Trans.log

further when I try to bring database TSN online  it gives error as below

2> go
1> online database TSN
Unable to obtain exclusive access to database 'TSN'.

any documents , reference or link is highly  appreciated

Regards
Mohammad

Hello Stephen,

Thanks.... I have open another Thread with Sybase replication Switch over method

http://scn.sap.com/thread/3495576

Regards

Mohammad

If this were a non-SAP environment the folks in this forum could give you suggestions on how to perform a switch for MSA and WarmStandby configurations.  And while the advice/suggestions would likely work in a SAP environment (at least technically speaking) I'm not sure how this would affect a) the SAP application management infrastructure or b) your licensing with SAP.

Since you're running SAP on ASE and using the various SAP 'tools' (eg, sap_status is a SAP-centric piece of code) to setup your replication environment, I'd recommend you visit SAP LT Replication Server (that group deals with SAP applications using Sybase's repserver) to see if the folks over there can give you more details on what options may be of use in your situation.

Thanks Mark..

I will post same question there.... but still I would appreciate any method on doing failover of Sybase ASE database from primary to secondary

Even I have raised incident with SAP , let see how it goes

Regards
Mohammad

Hi Mark,

your statement is not correct.

SAP LT Replication Server (SLT) is an own product without any relation to Sybase. It can be used for real time replication.

Best Regards,

Tobias

Tobias,

Thanks for the correction.

Any ideas/suggestions on where Mohammad should be directing his questions for SAP apps using Sybase replication server? (There are separate groups for SAP on ASE and non-SAP on ASE ... haven't found (yet) a separate group for SAP on Sybase repserver.)

Puh, I would expect that this is the right forum, because it is a question related to SRS.

Sorry no idea.

Best,

Tobias

Mohammad,

Sorry for the misdirection.

I can't find any other groups so I guess this is the 'right' place for SAP on Sybase repserver.

For general purpose comments on switching activities we'd need to know if your replication is setup using table-level replication (I kinda doubt this), MSA or WarmStandby.  Unless you've got documentation that answers this question you can run the following:

- if using WarmStandby then you'll need a logical connection that relates your PDS.PDB to your RDS.RDB; you can log into the repserver and run 'admin logical_status'; if this returns some records that include your PDS.PDB and RDS.RDB pairs then you're using warm standby

- if using MSA you can log into the (e)RSSD and run rs_helpdbrep and rs_helpdbsub; if you're using MSA the output from these commands will show the database repdef and subscription related to your databases

Otherwise you'll need to a) wait for someone with SAP-centric/replication server knowledge to read this thread or b) open a case with SAP tech support to see if they have some documentation on how to switch databases in a SAP application/replication environment.

Hi Mark,

Thanks for Input, I will run below commands and get back to you ....

Regards
Mohammad

Hi Mohammed,

In Business Suite on ASE, there is a Sybase tool used by the SAP tooling for setting up and administering replication.  That tool is called DR Agent - and is whose "sap_status path" output you are sharing in this thread.

Overview of the solution can be seen here:

http://scn.sap.com/community/services/blog/2013/10/03/sap-sybase-replication-server-hadr-high-availability-disaster-recovery-for-business-suite-on-ase

While the DR Agent will participate in the replication failover processing, it is the SAP toolset that invokes it, and also manipulate the SAP environment to fail-over the SAP application as well.

I'll try to find correct link to that detailed information.

Regards,
Stephen

Hi Mohammed,

Please see SAP note 1868514 that describes using SAP tool saphostctrl to perform the failover.

Regards,
Stephen