Assigning an IP address to a second interface on an Oracle Linux Instance

From Iwan
Jump to: navigation, search

This tutorial provides step-by-step instructions on assigning an IP address to a secondary network interface on an Oracle Linux instance. Whether you are expanding your network configuration or setting up a specific routing policy, adding and configuring a second interface is a critical skill.

You'll learn how to:

  1. Identify the available network interfaces.
  2. Configure the secondary interface with a static or dynamic IP address.
  3. Update network configuration files for persistence across reboots.
  4. Verify the configuration and ensure connectivity.

By the end of this guide, you’ll be able to extend the networking capabilities of your Oracle Linux instance confidently, enabling it to interact seamlessly across multiple subnets or networks.

-30012025-7.png

Objectives

In this tutorial, you will learn how to assign an IP address to a second network interface on an Oracle Linux instance. By the end of this guide, you will:

  1. Understand the configuration files involved in setting up a secondary network interface.
  2. Learn how to assign a static IP address to the second interface.
  3. Verify the connectivity and proper functionality of the newly configured interface.

Prerequisites

  • VCN
  • 2 x Subnets

The Steps Summary

  • [ ] STEP 01 Create a new Instance
  • [ ] STEP 02 Add second vNIC to the Instance
  • [ ] STEP 03 METHOD 01 Use OCI CLI (oci-utils) to assign an IP address to the second interface of an OCI Instance (using the oci-network-config command)
  • [ ] STEP 04 METHOD 02 Use OCI CLI (oci-utils) to assign an IP address to the second interface of an OCI Instance (using the ocid daemon)
  • [ ] STEP 05 METHOD 03 Use the OCI_Multi_VNIC_Setup script
  • [ ] STEP 06 METHOD 04 Create the interface config file manually for the new VNIC in the /etc/sysconfig/network-scripts/ folder

For a comprehensive and practical learning experience, start with Method 1 or Method 2, as these leverage OCI-native tools and provide a straightforward setup. As you gain familiarity, explore Method 3 for scripting and automation, and finally, try Method 4 for advanced manual configurations.

STEP 01 - Create a new Instance

In this section, we will create a new instance in the OCI cloud environment.

  • Use the hamburger menu to navigate to through Compute > Instances.
  • Click on the Create instance button.

770bd321ece2e5e324ef6c7dfda70ab0.png

  1. Specify a name for the instance (we will use sriov-test-01)
  2. Scroll down.

2926194955dca9448cadfefe535339eb.png

  1. We will change the shape:
  2. to VM.STandard.E4.Flex (but if you want to use E5 you can).
  3. Scroll down.

06b6b5359ca9492884851f3a79a0f950.png

  1. We are using an existing VCN.
  2. Select the VCN (we are using the oke VCN).
  3. Select the existing Subnet (for the primary vNIC).
  4. Scroll down.

Aaae532a26b4ef57c7892114a7d1afa8.png

  1. For (first) vNIC interface IP address we select to Automatically assign the private IPv4 address.
  2. As we selected a public subnet for the primary interface we select 'Automatically assign the public IPv4 address*
  3. Scroll down.

02d971bdc56d635b10861dc2b63f52ab.png

  1. We will use/upload existing (previously generated) public and private SSH keys.
  2. Click on Browse to select the existing public key.
  3. Verify if the existing public key is selected for upload.
  4. Click on the Create button.

2ca6ce0e11724f24d61a3b681cac4571.png

  1. If the instance is deployed successfully the status will be showing RUNNING.
  2. Notice that the primary VNIC is configured and the IP address is assigned automatically.

Cfa8e622a449edd589c943d19a4b7ffd.png

STEP 02 - Add second vNIC to the Instance

  1. Scroll down (after we created the instance) and click on Attached VNICs.
  2. Notice that there in only one VNIC and we renamed this to sr-iov-test-01 (the default name that OCI provides is different).
  3. Click on the Create VNIC button.

931f7cbd81dc0e2994462cf9ef2425e9.png

  1. Specify a name for the second VNIC (we will use secondary-vnic)
  2. Select the VCN (we are using the oke VCN).
  3. Select the existing Subnet (for the secondary vNIC) This is a different Subnet than that the Primary VNIC is attached to.
  4. Enable the checkbox to Use network security groups.
  5. We creates a Network Security Group up front that allows all ingress and egress traffic. Select this group.
  6. Scroll down.

Step 5 and 6 are optional.

Bea8c8e91a254c04ba886d5e08b8d40b.png

  1. For (second) vNIC interface IP address we select to Automatically assign the private IPv4 address.
  2. Click on the Save changes button.

4d158b6729b16e6c9e6f89fdb4b5c9b8.png

  • Notice that the second VNIC is now created and attached to the instance and the other Subnet.

Cef103d82489b4369e106097588379fa.png

Note

Even tough we have selected the option that the second VNIC should automatically get an IP address from OCI the Operating System (Oracle Linux) will not configure the IP address on the VNIC. The IP address is reserved by the OCI console, but is not configured yet.

The configuration of the IP address on the second VNIC can be done with multiple methods.

  1. Use OCI CLI (oci-utils) to assign an IP address to the second interface of an OCI Instance (using the [oci-network-config command])
  2. Use OCI CLI (oci-utils) to assign an IP address to the second interface of an OCI Instance (using the [ocid daemon])
  3. Use the [OCI_Multi_VNIC_Setup script]
  4. Create the interface config file manually for the new VNIC in the /etc/sysconfig/network-scripts/ folder.

In the steps below we will test them out one by one.

STEP 03 - METHOD 01 - Use OCI CLI oci-utils to assign an IP address to the second interface of an OCI Instance

Method 1 - Test with a new clean Instance sriov-test-04

I have created a new instance named "sriov-test-04" that I will be using to configure the IP address for the second VNIC using the OCI CLI (oci-utils). I will assign an IP address to the second interface of my OCI Instance (using the [oci-network-config command]).

Prerequisites

  • Provisioned a new Instance
  • Added a second VNIC

Configure IP address on the second interface

  1. Issue the ip a command to retrieve the IP address from all the interfaces.
  2. Notice the ens3 interface.
  3. Notice that the ens3 interface has an IP address configured.
  4. Notice the ens5 interface.
  5. Notice that the ens5 interface has NO IP address configured.
  6. Issue the route -n command to retrieve all the routes (and default gateways) for all interfaces.
  7. Notice that there is only a gateway available for the ens3 interface.

5ee240499b895b29086a922b27b75de4.png

The following commands where used to "test" if the oci-utils commands are working out of the box.

  1. Issue the command sudo oci-compartementid to retrieve the OCID for the compartment where the Instance is part of.
  2. Issue the command sudo oci-instanceid to retrieve the OCID for the Instance.

Here we have proven that oci-utils commands are working out of the box.

6103174a7c2d583876236f1371c1fc69.png

  1. Issue the command sudo oci-network-config show to retrieve all network configuration from the instance.
  2. Notice the error: Cannot show information: Failed to get API session. The information regarding the VNICS of the instance will not be provided on Operating System Level and not on OCI level.
  3. Notice the Operating System Level information on the ens3 interface.
  4. Notice the Operating System Level information on the ens5 interface.
  5. Notice the ADD flag for the interface ens5.

F72759f8a30dd5bebda8ba2e117769b6.png

To configure the ens5 interface that currently is flagged with ADD we need to issue the OCI CLI command sudo oci-network-config configure.

This is not possible as we will get the same error when we issue the command oci-network-config show that is Cannot show information: Failed to get API session.

Why are we getting this error?

  • A VCN is not an object owned by an instance, which means the instance cannot directly access VCN information without proper authentication.
  • To enable the instance to fetch VCN details from the API, you must create a dynamic group that includes the instance and assign a policy granting the necessary permissions.

To enable an Oracle Cloud Infrastructure (OCI) instance to fetch Virtual Cloud Network (VCN) details via API, the following steps are typically required:

  1. Create a Dynamic Group: This dynamic group includes the instance(s) that require access to the API. You define the group using rules based on attributes like instance OCID or compartment.
  1. Assign Permissions with a Policy: You need to create an Identity and Access Management (IAM) policy that grants the dynamic group permission to read or manage VCN-related resources. For example:
allow dynamic-group <dynamic-group-name> to read virtual-network-family in compartment <compartment-name>
  1. Use Instance Principals for Authentication: Instances in the dynamic group authenticate directly using their identity, allowing secure API access without embedding credentials in the application. This method simplifies security management while maintaining robust access control.

Let's try it out.

  1. Click on the hamburger menu by navigating through Compute > Instance and click on the compute instance you want to issue the commands sudo oci-network-config show and sudo oci-network-config configure on.
  2. Click on show to display the Instance OCID value.
  3. Click on copy the copy the Instance OCID value.

Fce1b4c5f487448bca90fba88a4d3d88.png

  1. Click on the hamburger menu by navigating through Identity > Domains > Default domain > Dynamic groups.
  2. Click on the Create dynamic group button.

4c90f08de9f643654acd72d5ca2fc303.png

  1. Specify a name for the Dynamic group (we will use sriov-test-04)
  2. Click on rule builder.

821cef75e4a9ad85b338d70be42146c2.png

  1. Match instances with Instance OCID.
  2. Paste the the OCID value of the instance (that we just copied)
  3. Click on the Add rule button.

Add22ff2ef1a8af22c58af2f28badb7a.png

  1. Notice that a new rule is added to the group.
  2. Click on the Create button.

Ddd6a91a08163f7e9273bd0b755d06a6.png

Now that we have the group where the instance is now part of, we need to create a policy inside the compartements where the Instance is part of to allow that instances in this group are allowed to perform operations on VCN objects (like subnets). In our case this is oraseemeaocids4 > NETWORK > Iwan

  1. Click on the hamburger menu by navigating through Identity > Compartments
  2. Click on the root compartment (in our case this is oraseemeaocids4).

8029ad696636e2bfa5d86911313dc79e.png

  1. Notice that we are now in the child compartments.
  2. Click on the next child compartment (in our case this is NETWORK).

7a045daf6886a93ac557ca14ca409d61.png

  1. Notice that we are now in the next child compartments.
  2. Click on the next child compartment (in our case this is Iwan).

328a86e7298b2d6753b0d8d30e3faf6c.png

  1. Make sure the right compartment is selected.
  2. Click on Policies.
  3. Click on the Create Policy button.

Ebfaea69c19e9f8afe06de43aea39cee.png

  1. Specify a name for the Policy (we will use allow-sriov-test-04-to-vcn).
  2. Specify a description for the Policy (we will use allow-sriov-test-04-to-vcn).
  3. Toggle the Show manual editor to on.
  4. Specify the following policy allow dynamic-group sriov-test-04 to manage all-resources in compartment Iwan
  5. Click on the Create button.

54864ad428f8797a4a58b2b221979f90.png

Notice the new Policy statement inside the newly created policy.

3ce8877a649e24b4084e11fce76a45ab.png

Now that we have allowed the instance to perform API calls (through OCI CLI) to the VCN, lets jump back on the instance.

  1. Issue the ip a command to retrieve the IP address from all the interfaces.
  2. Notice that the ens5 interface has NO IP address configured.
  3. Issue the command sudo oci-network-config show to retrieve all network configuration from the instance on OCI level. Notice that the error is now gone.
  4. Notice the OCI Level information on the ens5 interface and the ADD flag for the interface ens5.
  5. Notice the Operating System Level information on the ens3 and ens5 interfaces.

Ec566da17cbd744816b177fb4ba687f3.png

Before we issue the sudo oci-network-config configure command to configure the IP address of the second VNIC we need to create a new directory (/etc/iproute2) and a new file (/etc/iproute2/rt_tables). This is required so that the system can correctly configure the routing and default gateway for this new interface.

The /etc/iproute2/rt_tables file is used to define and manage multiple routing tables in Linux, enabling policy-based routing. It maps human-readable names to routing table numbers, allowing advanced setups like routing specific traffic through different gateways, interfaces, or ISPs.

  • Create a new /etc/iproute2directory with the command sudo mkdir -p /etc/iproute2
  • Create the rt_tables file with the command sudo touch /etc/iproute2/rt_tables
  1. Issue the sudo oci-network-config configure command to configure the IP address of the second VNIC.
  2. Notice the Configured message.
  3. Issue the command sudo oci-network-config show to retrieve all network configuration from the instance on OCI level.
  4. Notice the OCI Level information on the ens5 interface and the ADD flag for the interface ens5 is now gone.
  5. Issue the ip a command to retrieve the IP address from all the interfaces.
  6. Notice that the ens5 interface now has an IP address configured.
  7. Issue the route -n command to retrieve all the routes (and default gateways) for all interfaces.
  8. Notice that the gateway is now available for the ens5 interface.

10d3cfd49088cece34143df305ceb5a3.png

  1. Issue the command more /etc/iproute2/rt_tables to verify it a new entry is made.
  2. Notice the 11 ort entry.
  3. Issue the command ip route show table 11 to review the routing table for this mapping.
  4. Notice that there is a default route for the interface ens5 the new second VNIC on the instance.

1fa27c02d5663ab286f05530fdc4ea24.png

STEP 04 - METHOD 02 - Use OCI CLI oci-utils to assign an IP address to the second interface of an OCI Instance using the ocid daemon

Method 2 Test with a new clean Instance sriov-test-01

I have created a new instance named "sriov-test-01" that I will be using to configure the IP address for the second VNIC using the [ocid daemon].

Prerequisites

  • Provisioned a new Instance
  • Added a second VNIC

Configure IP address on the second interface

  1. Issue the ip a command to retrieve the IP address from all the interfaces.
  2. Notice that the ens5 interface has NO IP address configured.

D3e9f6cd91e556d4fcf04e02ce26234a.png

  1. Issue the route -n command to retrieve all the routes (and default gateways) for all interfaces.
  2. Notice that there is only a gateway available for the ens3 interface.

1e0f2b1baec9c7525f3fd65a6880c4ab.png

  1. Issue the (filtered) command ip -f inet -o addr to show all configured IP addresses on all interfaces.
  2. Notice that there is only configured interface and IP address is the ens3 interface.

16d2b9242bfc00d486e0effc14ab11e3.png

  1. Issue the command systemctl list-unit-files ocid.service to verify if the OCID daemon is enabled. Notice that the
  2. Notice that the OCID daemon is disabled.
  3. Issue the command sudo systemctl start ocid to start the OCID daemon.
  4. Issue the command sudo systemctl status ocid to verify the status of the the OCID daemon.
  5. Notice that the OCID daemon is active and running.

Ffe3c93fe1e3bd869601b2dfe6f783f3.png

  1. Issue the (filtered) command ip -f inet -o addr to show all configured IP addresses on all interfaces.
  2. Notice that ens5 interface is now configured and has an IP address.

Ed8c3b8a141705a336e2cc0e84978594.png

  1. Issue the ip a command to retrieve the IP address from all the interfaces.
  2. Notice that the ens5 interface now has an IP address configured.

753d3e772a8925b953bf00db23178568.png

  1. Issue the route -n command to retrieve all the routes (and default gateways) for all interfaces.
  2. Notice that the gateway is now available for the ens5 interface.

Eb8e80db2c416caef479530dc1b47577.png

  • To make sure the OCID service is automatically starting after a reboot issue this command sudo systemctl enable ocid

810dc6c1083be04a4a285f8a51713509.png

Just like we did in method 1 we also need to create a new directory (/etc/iproute2) and a new file (/etc/iproute2/rt_tables). This is required so that the system can correctly configure the routing and default gateway for this new interface.

The /etc/iproute2/rt_tables file is used to define and manage multiple routing tables in Linux, enabling policy-based routing. It maps human-readable names to routing table numbers, allowing advanced setups like routing specific traffic through different gateways, interfaces, or ISPs.

  • Create a new /etc/iproute2directory with the command sudo mkdir -p /etc/iproute2
  • Create the rt_tables file with the command sudo touch /etc/iproute2/rt_tables
  • Issue the sudo reboot command

61243ccbecb4ce8adb200cdba4f1090d.png

  1. Issue the command more /etc/iproute2/rt_tables to verify it a new entry is made.
  2. Notice the 11 ort3 entry.
  3. Issue the command ip route show table 11 to review the routing table for this mapping.
  4. Notice that there is a default route for the interface ens5 the new second VNIC on the instance.

3d390f2bac67636e95796a16ccb0c8a0.png

STEP 05 - METHOD 03 - Use the OCI_Multi_VNIC_Setup script

Method 3 Test with a new clean Instance sriov-test-02

I have created a new instance named "sriov-test-02" that I will be using to configure the IP address for the second VNIC using the [OCI_Multi_VNIC_Setup script].

Prerequisites

  • Provisioned a new Instance
  • Added a second VNIC

Configure IP address on the second interface

  1. Issue the ip a command to retrieve the IP address from all the interfaces.
  2. Notice that the ens5 interface has NO IP address configured.
  3. Issue the route -n command to retrieve all the routes (and default gateways) for all interfaces.
  4. Notice that there is only a gateway available for the ens3 interface.

49b15834573adcef3e6853590595e36f.png

  1. Download the OCI_Multi_VNIC_Setup.sh script with the curl command curl -o OCI_Multi_VNIC_Setup.sh https://raw.githubusercontent.com/Olygo/OCI_Multi_VNIC_Setup/refs/heads/main/OCI_Multi_VNIC_Setup.sh
  2. Use the command ls -l to list all the files.
  3. Notice that the script CI_Multi_VNIC_Setup.sh is downloaded sucessfully.
  4. Make the downloaded script executable with the command chmod +x ./OCI_Multi_VNIC_Setup.sh
  5. Execute the script with the command sudo ./OCI_Multi_VNIC_Setup.sh

F69dd66399dc8785fce15c7927682cd8.png

Before we continue with the script we first need to gather some information from the OCI console that we need to use as the input when we run the script.

  1. Go the the Instance details (in the OCI Console) and make a note of the IP address of the primary inteface.
  2. Scroll down.

Ace64c85c4042e8c56103bed3440beb9.png

  1. Click on Attached VNICs.
  2. Click on the second VNIC.

Ead0aa686b87c34d17122ec7b556b985.png

  1. Make a note of the MAC address of the second VNIC.
  2. Make a note of the IP address of the second VNIC.
Note

Even tough the IP address is not configured on the Instance Operating System, the IP address is already reserved by the OCI Management Console.

04c7eaf389da8da4d060835061221a55.png

Based on the IP address and subnet information we need to figure out what the default gateway is for that subnet as the script will ask for this information.

Note

The OCI console for not provide any details of the default gateway, but this is typically the first usable IP address in the subnet.


With this information we have the following information gathered:

Primary interface (ens3) Secondary interface (ens5)
IPv4 subnet: 10.0.0.0/29 10.0.3.0/27
IPv4 address: 10.0.0.4 10.0.3.5/27
IPv4 gateway: 10.0.0.1 10.0.3.1
MAC address: Not required for script 02:00:17:00:61:01
  1. Specify the primary VNIC name as ens3.
  2. Specify the primary VNIC IP as 10.0.0.4.
  3. Specify the primary VNIC Gateway as 10.0.0.1.
  4. Specify the secondary VNIC name as ens5.
  5. Specify the secondary VNIC IP as 10.0.3.5/27 with full CIDR Notation.
  6. Specify the secondary VNIC MAC address as 02:00:17:00:61:01.
  7. This interface will be part of a different subnet so type in y here.
  8. Specify the secondary VNIC Gateway as 10.0.3.1.

8c04027ac08ce41986dd2e3864d6381d.png

  1. Verify the information and if this is all correct confirm with y.
  2. Confirm the settings again with y.
  3. Notice that the script will start the interface configuration and will provide status updates while doing this.

A7119278e818a0df5ec2d0e51d001575.png

The full output of the status messages are given below: 

Summary of your configuration:
* The primary Vnic name is:            ens3
* The primary Vnic IP is:              10.0.0.4
* The primary Vnic GW is:              10.0.0.1
* The secondary Vnic name is:          ens5
* The secondary Vnic IP with CIDR is:  10.0.3.5/27
* The secondary Vnic MAC is:           02:00:17:00:61:01
* The secondary Vnic GW is:            10.0.3.1

Do you confirm these settings ? (y)es/(n)o/(q)uit: y

User settings confirmed...


Summary of your configuration:
* The primary Vnic name is:            ens3
* The primary Vnic IP is:              10.0.0.4
* The primary Vnic GW is:              10.0.0.1
* The secondary Vnic name is:          ens5
* The secondary Vnic IP with CIDR is:  10.0.3.5/27
* The secondary Vnic MAC is:           02:00:17:00:61:01
* The secondary Vnic GW is:            10.0.3.1

Do you confirm these settings ? (y)es/(n)o/(q)uit: y

User settings confirmed...

 ''''' Checking if NetworkManager CLI (nmcli) is installed '''''

nmcli is already installed.

 ''''' Configuring NetworkManager '''''
NetworkManager is already enabled.
NetworkManager is already running.
NetworkManager is configured and running.

 ''''' Check and install iproute if not installed '''''
Package 'iproute' is already installed. Proceeding.

 ''''' Ensure /etc/iproute2 directory exists '''''
Directory '/etc/iproute2' does not exist. Creating it...
Directory '/etc/iproute2' created successfully.

 ''''' Ensure /etc/iproute2/rt_tables file exists '''''
File '/etc/iproute2/rt_tables' does not exist. Creating it...
File '/etc/iproute2/rt_tables' created successfully with a default header.

 ''''' Starting ens5 configuration and persistence setup '''''

 ''''' Configure secondary VNIC with NetworkManager '''''
Connection 'ens5' (582dd2d7-3a3e-45b9-8289-ce3d474f4b74) successfully added.
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3)

 ''''' Enable IP forwarding and configure reverse path filtering '''''
= Enable IP forwarding =
net.ipv4.ip_forward=1

= Configure reverse path filtering =
net.ipv4.conf.all.rp_filter=2
net.ipv4.conf.default.rp_filter=2
net.ipv4.conf.ens3.rp_filter=2
net.ipv4.conf.ens5.rp_filter=2

 ''''' Apply sysctl settings immediately '''''
kernel.unknown_nmi_panic = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.all.rp_filter = 2
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.ens3.rp_filter = 2
net.ipv4.conf.ens5.rp_filter = 2

 ''''' Adding custom routing tables '''''
100 vnic_1
200 vnic_2

 ''''' Adding IP rules and routes '''''

 ''''' Creating policy routing persistence script '''''

 ''''' Creating systemd service for policy routing '''''
Created symlink /etc/systemd/system/multi-user.target.wants/policy-routing.service → /etc/systemd/system/policy-routing.service.

 ''''' Testing connectivity '''''
PING 10.0.0.4 (10.0.0.4) 56(84) bytes of data.
64 bytes from 10.0.0.4: icmp_seq=1 ttl=64 time=0.059 ms
64 bytes from 10.0.0.4: icmp_seq=2 ttl=64 time=0.037 ms
64 bytes from 10.0.0.4: icmp_seq=3 ttl=64 time=0.031 ms
64 bytes from 10.0.0.4: icmp_seq=4 ttl=64 time=0.027 ms

--- 10.0.0.4 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3073ms
rtt min/avg/max/mdev = 0.027/0.038/0.059/0.013 ms
PING 10.0.3.5 (10.0.3.5) 56(84) bytes of data.
64 bytes from 10.0.3.5: icmp_seq=1 ttl=64 time=0.026 ms
64 bytes from 10.0.3.5: icmp_seq=2 ttl=64 time=0.022 ms
64 bytes from 10.0.3.5: icmp_seq=3 ttl=64 time=0.023 ms
64 bytes from 10.0.3.5: icmp_seq=4 ttl=64 time=0.035 ms

--- 10.0.3.5 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3069ms
rtt min/avg/max/mdev = 0.022/0.026/0.035/0.007 ms
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.098 ms
64 bytes from 10.0.0.1: icmp_seq=2 ttl=64 time=0.097 ms
64 bytes from 10.0.0.1: icmp_seq=3 ttl=64 time=0.067 ms
64 bytes from 10.0.0.1: icmp_seq=4 ttl=64 time=0.060 ms

--- 10.0.0.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3069ms
rtt min/avg/max/mdev = 0.060/0.080/0.098/0.019 ms

 ''''' Network configuration completed '''''

IP rules:
0:	from all lookup local
32764:	from 10.0.3.5 lookup vnic_2
32765:	from 10.0.0.4 lookup vnic_1
32766:	from all lookup main
32767:	from all lookup default

user $ nmcli device show ens5

GENERAL.DEVICE:                         ens5
GENERAL.TYPE:                           ethernet
GENERAL.HWADDR:                         02:00:17:00:61:01
GENERAL.MTU:                            9000
GENERAL.STATE:                          100 (connected)
GENERAL.CONNECTION:                     ens5
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/ActiveConnection/3
WIRED-PROPERTIES.CARRIER:               on
IP4.ADDRESS[1]:                         10.0.3.5/27
IP4.GATEWAY:                            10.0.3.1
IP4.ROUTE[1]:                           dst = 10.0.3.0/27, nh = 0.0.0.0, mt = 101
IP4.ROUTE[2]:                           dst = 0.0.0.0/0, nh = 10.0.3.1, mt = 101
IP4.ROUTE[3]:                           dst = 0.0.0.0/0, nh = 10.0.3.1, mt = 0, table=200
IP6.ADDRESS[1]:                         fe80::70ba:d30f:3dac:6462/64
IP6.GATEWAY:                            --
IP6.ROUTE[1]:                           dst = fe80::/64, nh = ::, mt = 1024

 ''''' Configuration complete! ens5 is set up and all settings are persistent '''''

[opc@sriov-test-02 ~]$
  1. Issue the ip a command to retrieve the IP address from all the interfaces.
  2. Notice that the ens5 interface now has an IP address configured.
  3. After issuing the route -n command to retrieve all the routes (and default gateways) for all interfaces notice that the gateway is now available for the ens5 interface.

69796d8fa6dc948c542f30cb51ce2e87.png

Note

Configuring an IP address and making sure routing works correctly are two different things! Fortunately the script will also configure the routing part for us. Pay attention to the specific outputs of the script below:

Here the script will create the /etc/iproute2 directory and will create the rt_tables file. 

''''' Ensure /etc/iproute2 directory exists '''''
Directory '/etc/iproute2' does not exist. Creating it...
Directory '/etc/iproute2' created successfully.

 ''''' Ensure /etc/iproute2/rt_tables file exists '''''
File '/etc/iproute2/rt_tables' does not exist. Creating it...
File '/etc/iproute2/rt_tables' created successfully with a default header.

Here the script will add the necessary entries into the rt_tables file. 

''''' Adding custom routing tables '''''
100 vnic_1
200 vnic_2

Here the script will add the IP routing rules to allow policy based routing on the instance and will also enable the service for policy based routing which is required if you are have an instance with multiple VNICs. 

''''' Adding IP rules and routes '''''

 ''''' Creating policy routing persistence script '''''

 ''''' Creating systemd service for policy routing '''''
Created symlink /etc/systemd/system/multi-user.target.wants/policy-routing.service → /etc/systemd/system/policy-routing.service.
  1. Issue the command more /etc/iproute2/rt_tables to verify it a new entry is made.
  2. Notice the 100 vnic_1 and 200 vnic_2 entries.
  3. Issue the command ip route show table 100 to review the routing table for this mapping and notice that there is a default route for the interface ens3 the primary VNIC on the instance.
  4. Issue the command ip route show table 200 to review the routing table for this mapping and notice that there is a default route for the interface ens5 the new second VNIC on the instance.

0c72cef9afce4920a36c5fd581cfd553.png

STEP 06 - METHOD 04 - Create the interface config file manually

Method 4 Test with a new clean Instance sriov-test-03

I have created a new instance named "sriov-test-03" that I will be using to configure the IP address for the second VNIC by a definition for the new VNIC in the  /etc/sysconfig/network-scripts/ directory.

In this method we will be configuration almost all the steps that is automatically done by the script in [[https //github.com/Olygo/OCI_Multi_VNIC_Setup [#STEP 05 METHOD 03 Use the [OCI_Multi_VNIC_Setup script]]|Method 3 ]] for us.

Prerequisites

  • Provisioned a new Instance
  • Added a second VNIC

Configure IP address on the second interface

  1. Issue the ip a command to retrieve the IP address from all the interfaces.
  2. Notice that the ens5 interface has NO IP address configured.
  3. After issuing the route -n command to retrieve all the routes (and default gateways) for all interfaces
  4. Notice that there is only a gateway available for the ens3 interface.

Fcafc44bb5be1d01c83b144fac89ae15.png

  • Retrieve the IP address of the second VNIC by navigating to the Instance > Attached VNICS and select the second VNIC.
  • Make a note of the IP address that is assigned by the OCI Management Console.

84496a0b9852425c0cc299c14d910f70.png

  • Browse to the /network-scripts folder using the command cd /etc/sysconfig/network-scripts/
  • Create a new file in the /etc/sysconfig/network-scripts/ folder with the name ifcfg-ens5 using the command sudo nano ifcfg-ens5.

A4969d3860d87cb346508a5229f5c02d.png

  • Configure the ens5 interface with the following parameters:
NAME="ens5"  
DEVICE="ens5"  
IPADDR=10.0.3.19  
NETMASK=255.255.255.224
GATEWAY=10.0.3.1
ONBOOT=yes
  • Notice that the IP address is the IP address we retrieved from the management console.
  • The subnet is retrieved from the subnet inside the VCN that we have connected this interface to.
  • The Gateway IP address is the first IP address in the Subnet.

204e3a521e7c58f4da47a5e3beef5bf3.png

  • Save the config file ifcfg-ens5 with the nano editor.
  1. List the files in the /etc/sysconfig/network-scripts/ directory with the command ls -l.
  2. Notice that that there is an interface configuration file for ens3.
  3. Notice that that there is an interface configuration file for ens5.

1473df77fc29c83d11b880300f56f82a.png

  1. Verify the content of the ifcfg-ens5 file with the command more ifcfg-ens5.
  2. Make sure the parameters are all correctly configured.

9081dff973083226abc08c86f820cb74.png

  1. Issue the ip a command to retrieve the IP address from all the interfaces.
  2. Notice that the ens5 interface still has NO IP address configured.

3250e052a1ba1c8ecdb4e73082d9691d.png

  • Reboot the instance with the sudo reboot command.

75ff1894493d7dab7eea07d14f07965f.png

  1. Issue the ip a command to retrieve the IP address from all the interfaces.
  2. Notice that the ens5 interface now has an IP address configured.
  3. After issuing the route -n command to retrieve all the routes (and default gateways) for all interfaces notice that the gateway is now available for the ens5 interface.

B834c93bd80ffed95b0f82065c8e399d.png

Note

Configuring an IP address and making sure routing works correctly are two different things! In order to configure routing follow the steps below.

The /etc/iproute2/rt_tables file is used to define and manage multiple routing tables in Linux, enabling policy-based routing. It maps human-readable names to routing table numbers, allowing advanced setups like routing specific traffic through different gateways, interfaces, or ISPs.

  • Create a new /etc/iproute2directory with the command sudo mkdir -p /etc/iproute2
  • Create the rt_tables file with the command sudo nano /etc/iproute2/rt_tables
  • Configure the following parameters in the rt_tables file.
100 vnic_1
200 vnic_2
  • Add the static default route for the ens5 interface with the command sudo ip route add default via 10.0.3.1 dev ens5 table 200
  • Create the service file to enable the service for policy based routing which is required if you are have an instance with multiple VNICs with the command sudo nano /etc/systemd/system/policy-routing.service.
  • Configure the following parameters in the policy-routing.service file.
[Unit]
Description=Setup Policy Routing for Multiple VNICs
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/setup_policy_routing.sh
RemainAfterExit=true

[Install]
WantedBy=multi-user.target
  • Make the service file for policy based routing executable with the command sudo chmod +x /etc/systemd/system/policy-routing.service.
  • Start the service for policy based routing with the command sudo systemctl start policy-routing.service.
  • Make sure the service for policy based routing is loaded and active with the command sudo systemctl status policy-routing.
[opc@sriov-test-03 ~]$ sudo systemctl status policy-routing
● policy-routing.service - Setup Policy Routing for Multiple VNICs
   Loaded: loaded (/etc/systemd/system/policy-routing.service; enabled; vendor preset: disabled)
   Active: active (exited) since Wed 2024-12-11 09:07:16 GMT; 4 weeks 1 days ago
  Process: 2415 ExecStart=/usr/local/bin/setup_policy_routing.sh (code=exited, status=0/SUCCESS)
 Main PID: 2415 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 99891)
   Memory: 0B
   CGroup: /system.slice/policy-routing.service

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
[opc@sriov-test-03 ~]$
  • Make sure the service for policy based routing will be starting when the instance is (re)starting with the command sudo systemctl enable policy-routing.service.
  • Reboot the instance with the command sudo reboot.
  1. Issue the command more /etc/iproute2/rt_tables to verify it a new entry is made.
  2. Notice the 100 vnic_1 and 200 vnic_2 entries (that we manually configured before).
  3. Issue the command ip route show table 100 to review the routing table for this mapping and notice that there is a default route for the interface ens3 the primary VNIC on the instance.
  4. Issue the command ip route show table 200 to review the routing table for this mapping and notice that there is a default route for the interface ens5 the new second VNIC on the instance.

Bfdbca14142e0f1eb0e40365c61e158e.png

Conclusion

Choosing the right method to configure a second interface on an OCI Linux instance depends on your requirements, technical expertise, and desired level of control. If you are looking for a straightforward and efficient setup, Methods 1 and 2, which use the oci-utils package and Oracle-native tools, are highly recommended. These methods are ideal for users who want to leverage OCI’s built-in functionality without diving into manual configurations or external scripts.

For scenarios where automation and scalability are critical, Method 3, utilizing the OCI_Multi_VNIC_Setup script, provides a flexible solution. It is especially useful when managing multiple interfaces across different instances, offering time savings and consistency in configuration.

Meanwhile, Method 4, the manual configuration approach, provides the highest level of customization. This method is best suited for advanced users or specific environments where automation tools are either unavailable or not preferred. It also serves as an excellent way to understand the inner workings of networking on OCI Linux instances.

By understanding and mastering these methods, you gain the flexibility to configure secondary interfaces in a variety of scenarios, from basic setups to complex multicloud architectures. With the right approach, you can ensure reliable and efficient network configurations tailored to your OCI workloads.

Retrieved from "http://www.iwanhoogendoorn.nl/index.php?title=Assigning_an_IP_address_to_a_second_interface_on_an_Oracle_Linux_Instance&oldid=7175"