At a high-level view the setup would look like:
a - DB2 source database (DB2): transactional activity written to DB2 log
b - Replication Agent for DB2 (RA; a stand-alone, java-based program) reads transactions from DB2 log and sends said transactions to the Repserver (RS) [think of the RA as a translator; in this case it translates DB2 log activity into a language the RS can understand]
c - Repserver processes the transactions and submits them to ExpressConnect for Oracle (ECO; a java-based plugin to the RS software)
d - ECO applies the transactions to the Oracle target database (ORA) [think of the ECO as another translator; in this case it translates RS activity into a language ORA can understand]
So:
DB2 -> RA -> RS -> ECO -> ORA
You supply the DB2 and ORA components.
Sybase supplies the RA, RS and ECO components.
The link Kimon has provided goes into more detail on the above items.
-------------
While Repserver is highly customizable, the high level of customization can quickly leave a newbie drowning in the technical details. And while setting up Sybase-to-Sybase replication isn't too hard (eg, there are no standalone translation modules - RA, ECO - needed), there's an additional level of complexity when using Repserver with non-Sybase databases.
NOTE: While I work with the Replication server product, I do not work with any of the SAP applications, so fwiw ...
SAP is putting a lot of effort into providing GUIs/tools for installing/configuring/maintaining the various Sybase products, especially when they're used in a SAP application environment. This includes support for the Repserver product.
If you're seriously thinking about using Repserver you may also want to investigate what tools the SAP folks are providing for using Repserver to move transacions out-of/into SAP databases. [The objective being to see if any of these tools could make life a little easier for a newbie trying to setup a Repserver environment.]
I'd recommend visiting SAP LT Replication Server to see if the folks over there can give you more details on what options may be of use in your situation.