Category Archives: RouterOS

MikroTik RouterOS ZTE MF90 bug in 6.36 up to 6.43.7, Huawei E5573 & E5673, Movimax MV003 support

I have never installed any USB 3G/4G modem on my MikroTik RouterBOARD devices, despite most of them having a USB port available.  Finally I came across 2 situations where I had to deploy 4G USB modem as primary link at an event venue and as backup Internet connection at home.

I have 2 unused Bolt4G’s MF90 modems in unlocked state so I wanted to use them to save money from buying new modems.  I have many MikroTik RouterBOARD devices at my disposal, 2 hap ac lite, 1 hap ac^2, and 1 RB951Ui-2HnD.  This time I need to plug the modem into the hap ac lite and RB951Ui-2HnD running RouterOS 6.43.4 (the latest OS at the time).  There are tons of materials on MF90 and MikroTik routers because it was such a popular device at its time, which led me to think it should not be difficult to achieve.  It should be a walk in the park, so people say.  Turns out I was wrong.

MF90 drivers would not load on any of the MikroTik RouterBOARD devices I own.  I checked /system resources usb print and I could see the MF90 there.  It was recognised but there was no new interface in the /interfaces lte menu.  I checked the logs, restarted the modem, swapped the 2 MF90s, replaced the USB cable, rebooted the MikroTik devices, still nothing!  I googled further but I couldn’t find why it wouldn’t work, until I came across this and this.  Apparently one person have reported this issue to the MikroTik forum back in September 2016, yes 2016.  More than 2 years ago.  Guess what, nobody from MikroTik team responded to this poor guy (sger on  sger mentioned that it had last worked for him on RouterOS 6.35.4 on his CRS125.  I immediately googled for that version of RouterOS (mipsbe package for the hap ac lite that I was using as a testing device) and downgraded the RouterOS.  Voila!  It immediately showed the lte1 interface, so sger is damn right about this.  Typical MikroTik bug I would say, I have seen a couple of times they lose older features to bugs like this.  I reached out to the kind guys at MikroTik support team, this time it went to Antons B.  One thing I like about MikroTik support is most of them know what they are talking about.  Another company who does well at this is cPanel.  Anyway back to topic, I knew I had to report this bug ASAP and they will gladly fix the issue because it was supported then and it should work ever since.

As always MikroTik support asked me to send my supout.rif file.  I had to disable my wireless package since 6.36 was the version where MikroTik introduced the new single wireless package, downgrading to 6.35.x would require me to disable wireless package before I could generate a supout.rif file for MikroTik support.  When it was enabled, the process to generate a supout.rif file would max out the CPU and end up crashing the process.  After sharing my supout.rif files on both the latest RouterOS 6.43.4 and 6.35.4, Antons came back with a new build of RouterOS 6.44beta32.  He told me to test this build and as mentioned above, it solved the problem flawlessly.  It took him only less than 3 days to come up with a new build.  This is why I love MikroTik, responsive and approachable.  If this were Cisco or Juniper, I don’t think I would be able to speak directly to their technical support team and most likely they would direct me to their partner’s support team which probably will never be able to solve the issue.  3 weeks later, MikroTik released 6.43.7 which is the latest stable build with the MF90 fix.

Now with the MF90 working, I get to achieve what I wanted to do.  However the MF90 has B08 firmware which is known to be locked to TDD2300 (B40). B05 firmware is said to support FDD2100 but it appears that nobody has done it (or rather share it publicly) so I ended up looking for new USB modems to take advantage of the FDD2100 band (where most of the national telcos operate their 4G services).  I did some research and Huawei’s devices seem to standout.  I considered XL Go Izi’s Movimax MV003 but I couldn’t find any reference stating that it’s usable on MikroTik (turns out it is working well with MikroTik after my colleague lent me hers to test).  Specs-wise it doesn’t say anything other than Qualcomm chip powered.   Since I’m not in a position to gamble right now, I took a safer choice in Huawei E5673.  Some references can be found on Google about it and it looks like a solid modem.  It is suitable for my first use case, however for the second use case I needed a AlwaysOn-like package for the backup Internet connection at home.  I considered Tri, but its performance on both 3G and 4G is just so so.  Telkomsel is expensive and thus is out of the question despite being the strongest candidate since they run both FDD2100 and TDD2300 LTE networks.  Indosat is probably the weakest of the top 3, so naturally it goes back to XL, my previous employer.  XL’s 3G and 4G services prove to be quite fast and reliable at home, but its long-term data package, branded XL Go Izi, requires XL Go-branded modems (E5573, E5577, MV003).  Since the recently purchased E5673 wouldn’t work with it, I ended up buying another one, an XL Go-branded Huawei E5573.  I did not want to risk buying E5577 nor MV003 because nobody has shared their experience getting them to work with MikroTik.  Anyway, after I received the E5573, I immediately started with a speedtest.  I got close to 30Mbps of download in the first attempt, 3X faster than my home FTTH Internet connection.  Now my backup is ready to pick up whenever the FTTH goes down.  Don’t hesitate to buy Huawei E5573, E5673, and Movimax MV003 if you wish to use them on MikroTik devices, I have tested all 3 and confirmed that they play nicely with MikroTik RouterOS!

VMware ESXi on’s dedicated server, additional IPv4 subnet, and WORKING native IPv6 connectivity

I have finally migrated my server from LeaseWeb to OVH.  LeaseWeb wouldn’t provide me with a reasonable offer after being a loyal customer for 3 years so I terminated my server there. I came across OVH’s 1-year anniversary for Asia Pacific servers (50% off for annual payment) and their offer was massively appealing so I immediately signed up. I am extremely impressed with their customer control panel. I would say it’s much better than SoftLayer and LeaseWeb if you like to configure things by yourself.  OVH may not have a perfect reputation but it seems to have improved significantly over the years. I’m quite pleased on their automatic provisioning that took minutes and the KVM-over-IP is very easy to use.

It took me a while to get up to speed but once I got a hold of it, things were quite straightforward.  I intend to run a VM-based router (MikroTik’s Cloud Hosted Router) and route the subnet through it and other VMs will be a part of this VM network.  I could “bridge” my VMware ESXi CHR VM to the physical network to take advantage of their very-reasonably-priced one-time-fee additional IPv4 blocks using a FailOver (FO) IP address.  You need 1 individual FO IP to route/”bridge” the additional IPv6 subnet(s).  Apply the OVH-generated VMware-accepted MAC address to your VM host’s virtual ethernet (in my case VMware ESXi) to get the “bridge” to work.  You will need to “bridge” every single IP to the OVH-generated MAC address in the subnet manually.

Once I got everything working as expected, I continued to configure native IPv6 as provided by OVH.  I read the available guides on and on Google (they have various guides on IPv6 make sure you read the right one for dedicated server), but I could never get it to work.  I got assigned the IPv6 range of 2402:1f00:8000:3cb::/64 but the guide states that I need to use 2402:1f00:8000:3ff:ff:ff:ff:ff as the gateway IPv6 address, which is unusual because such address is outside the /64 range.  2402:1f00:8000:3cb::/64 means 2402:1f00:8000:3cb:0:0:0:0 – 2402:1f00:8000:3cb:ffff:ffff:ffff:ffff.  Compare this range to the suggested gateway IPv6 address of 2402:1f00:8000:3ff:ff:ff:ff:ff (or 2402:1f00:8000:03ff:00ff:00ff:00ff:00ff), you can clearly see that the suggested IPv6 gateway address is outside the range.  This is the reason why this unusual network setup requires some effort to make it work properly.

I did not really take notice of this initially and wondered for hours why it wouldn’t work.  I opened a support ticket and the response was not helpful, the response only repeated the guide. I eventually used an IPv6 calculator to confirm my suspicion and it proved to be the culprit (because the gateway error was “unreachable” and “no route to host” when ping was used). I immediately tried changing the subnet mask to /56 instead of /64 because the assigned gateway IPv6 address is expected to be overridden with “ff” and changing it to /56 now makes the gateway IPv6 address within the network.  Remember, previously it was outside with the /64 range. Now the OS would accept the gateway IPv6 address and is reachable.  Ping also worked this time and immediately I had native IPv6 connectivity.  However this is not the right setup!

So, if you are an OVH customer and you cannot get IPv6 to work on MikroTik after reading their knowledge base articles , here is how:
The kind guy at MikroTik support (Martins S.) helped me to get this to work on my MikroTik setup.  The solution is actually very simple.  Basically once you have set up the IPv6 address on MikroTik, use multicast ping command: /ping FF02::2%ether1.  ether1 is the interface name connected to OVH.  Write down link-local addresses responding to the ping and expect to see a few replies for every ping request, including the device’s own link-local interface.  In my case I received 3 replies from 3 link-local addresses per ping, after eliminating my own device’s link-local address, I ended up with 2 link-local addresses.  These should be the link-local address of the actual router.  You can use this command to set up the IPv6 default route: /ip6 route add dst-address=”2000::/3″ gateway=”%ether1″. The new default route should immediately become active and you can use /tool trace to confirm your IPv6 connectivity on OVH’s network (assuming you have a working resolver configuration in place ;).

Good luck!

iptables mangle and NAT notes, etc.


PREROUTING in mangle table: alter routing (e.g. source-based routing)
FORWARD in mangle table: traffic shaping for tc (flowid)

In MikroTik RouterOS, /ip firewall nat:
srcnat chain = PREROUTING
dstnat chain = POSTROUTING

/ip firewall mangle:
prerouting chain = global-in
forward chain = global-out (support NAT traffic shaping, higher load on router)

Update: I used to think that global-in literally means global input, while global-out literally means global output.  Apparently I was mistaken.  The parent queues MUST be matched to the right parent, either global-in, global-out, global-total, or one of the network interfaces depending on the mangle rules.  Top-most queue parent (global-in, global-out, global-total) doesn’t decide whether it’s up/down.  The mangle rules do the direction decision trick, whether a packet is incoming/download or outgoing/upload.  This makes perfect sense now! (doh)  I kept wondering why half of my queues (download queues’ top-most parent was global-in) weren’t working, when all my mangle rules were in forward chain.  Obviously they wouldn’t, because global-in marks both direction in prerouting chain.

Update 2: mangle: download rules first then upload rules. global-in, connection mark: prerouting; packet mark: prerouting. global-out, connection mark: forward, packet mark: postrouting.


This didn’t make sense until I read the URL listed below under References.  Silly me!  Assumption is the root of most, if not all, problems indeed.


MikroTik RouterOS Interface Bonding

I have two separate Metro Ethernet links (via fiber optic) from the datacenter to the NOC. Each link is 10Mbps. I need to utilize both links (bonding) and make sure sure that if one of the links goes down (redundancy), I won’t lose half of my packets. Bonding and redundancy are my goals.

Initially I tried Cisco Catalyst’s EtherChannel feature to accommodate this need since I learned about EtherChannel when I was doing my CNAP. Unfortunately EtherChannel cannot fit in this scenario due to my Metro Ethernet provider’s network setup. They use Cisco Catalyst 3750 switches to aggregate customers links from each POP to their headquarters. My first attempt was to establish trunk mode EtherChannel (802.1q) with Cisco Catalyst 2950 on one side and Cisco Catalyst Express 500 on the other side. Later I noticed that this is not doable since trunking requires MTU size to be larger than 1500 (1504) when my provider strictly limits MTU size to 1500 and negotiation between my 2 switches to establish trunking wouldn’t work since my switches’ BPDU packets are “intercepted” by my provider’s switches. Basically my Cisco switches were trying to establish a VLAN trunk with my provider’s directly connected switches when my switches are  supposed to be negotiating to each other.

I consulted a few experienced people including an employee of the provider, and they told me to use access mode EtherChannel instead of the trunk mode EtherChannel. This is not possible with Cisco Catalyst Express 500, which only offers trunk mode EtherChannel. I bought a Cisco Catalyst 2960 to replace the Cisco Catalyst Express 500 hoping that access mode EtherChannel would work, it didn’t. Even if it did work, it wouldn’t be aware of link state changes since my switches do not connect directly to the fiber cables. There is a fiber-to-ethernet bridge for each side of each link, so my switches will always detect both links as always up as long as the bridges are up.

Since link states cannot be used as a measurement in this scenario, I had to find another way. MikroTik RouterOS offers not only bonding feature, but fail-over mechanism too! The fail-over mechanism uses ARP packets to detect link failures, it is far from perfect but at least it works.

I will add examples later, but for now have a look at this. Hopefully I will discuss EoIP and EoIP over PPTP too.


PPPoE Server HOWTO for MikroTik RouterOS 2.9

If you wish to run a PPPoE server, MikroTik RouterOS provides a convenient way to set one up in a few minutes (with built-in traffic shaping feature too!). Previously I used Fedora Core for my PPPoE servers, but I couldn’t find a working solution to keep ghost PPPoE sessions from bogging down my Linux server. I tried MikroTik DOM with RouterOS to replace my Linux-powered PPPoE servers, so far the results are very good.

Below is a mini guide that may be able to help you get your PPPoE server running in a few minutes using RouterOS.

First, make sure that your RouterOS server’s WAN connectivity has been properly configured. Remember that you need at least 2 network interface cards (NICs). This guide assumes that both NICs are ethernet — ether1 and ether2. If you haven’t set anything up on the new system, let me help you with the checklist: (based on my experience, the following issues are the most common)

  • Make sure that the Internet-facing NIC has an IP address assigned on it and the default gateway is set (/ip route add gateway=…)
  • If NAT is used, ensure that src-nat/masquerade firewall rule has been added (/ip firewall nat …) and it is working properly

Once you have verified the server’s connectivity, create a PPP profile (/ppp profile add name=”pppoe-profile” local-address= dns-server=… rate-limit=128k/128k). Every user account that uses this profile will get 128Kbps upload and download limit. If you wish to have different types of accounts (for example some customers pay for 256Kbps), create a new PPP profile (change the rate-limit attribute).

Next, create a user account assigned to the new PPP profile (/ppp secret add name=”andryan” password=”test” service=pppoe profile=”pppoe-profile” remote-address= When this user logs in successfully, this user gets assigned To dynamically assign IP addresses, there is an example here.

Finally, create a PPPoE server instance (/interface pppoe-server server add service-name=”pppoe1″ interface=ether2 one-session-per-host=yes default-profile=”pppoe-profile”) and enable it. Now your RouterOS PPPoE server is ready to answer PPPoE requests and authenticate your PPPoE clients. 🙂

Good luck!


MikroTik RouterOS — BGP

I have always wanted to learn about BGP. This time I got the honor chance to implement BGP for an ISP. This ISP has its own AS number and a /21 IP address block. This BGP setup is pretty simple because I’m only using 1 PC (with 3 ethernet cards and a MikroTik’s level 4 DOM) to interconnect with an Internet Exchange (IX) — NiCE — and a transit ISP (another transit will be added soon).

In MikroTik RouterOS version 2.9.42, BGP features are available for level 3 and above. You also need to enable routing-test package if you want more flexibility (BGP filtering features). The routing package, as of this version, only allows basic BGP features. I’m sure this will change later when they release RouterOS v3. Once you have enabled routing-test package, you will see new options (Routing – Filters in winbox, or /routing filters on CLI). This is very important when you have to peer with 2 or more ASes (specifically an IX and a transit which is not interested in getting IX’s routes for obvious reasons).

Since I have never configured BGP before, I caused a major problem when routes from each peer goes to another peer when they shouldn’t! I didn’t place any filter for BGP advertisements my BGP router sends to its peers. My transit ISP received the IX’s routes and the IX received full Internet routes feed my BGP router gets from the transit ISP. BGP is an exterior distance-vector routing protocol, it picks its best path by comparing the AS path lengths of every route it has. Advertising the whole Internet BGP feed to the local IX caused other ISPs’ routers participating in the IX to discover a shorter path of international routes going via my BGP router so the routers chose this new shorter path instead and outgoing traffic of these ISPs started to flow via my transit ISP! On the other hand, advertising IX routes to my transit was not a big problem since the path will be farther for their customers anyway, having to go via my BGP router first (that’s one AS further for the transit ISP’s customers to reach the IX routes, so BGP will not select it).

Fortunately I was able to spot the error immediately and placed BGP routing filters to include only my /21 IP block in the advertisements my BGP router sends to both the IX and transit. I also added an incoming BGP filter to discard a default route my transit IP includes in its BGP feed. This default route is not required since I get full BGP feed that is unfiltered.

Useful links: — Route filters examples — Important! Make sure that BGP neighbors are reachable via static routes for dynamic routes to be active (in the case of multihop BGP neighbors)
Cisco’s BGP Reference