Setting up wifi in rural area South Africa

Dear Linda and Mvu, following on what Steve points out, the first step is find out where the best internet option is and this is a combination complexity and costs.

I think the first option, and probably cheapest in the short term, is to find internet service providers in Richmond. They don’t need to be WISP, maybe Telkom or Mweb, or other offer fixed internet there. Talking to people there will give you a sense of what is the most reliable, cheapest, etc provider. Then, do the line of sight analysis that Steve is proposing. There is a tool by ubiquiti that allows you do Radio Planning easier than with Google Earth. https://link.ui.com . Look for Richmond, South Africa, and click on add PtP link. Then you can play with the two ends of the line and it will show you the line of sight profile. It also allows you to tweak the height of the WiFi devices in both ends. I don’t know exactly where your household is, but it looks like there options for direct line of sight with places in Richmond. If there are, then go and chat with households in that part of Richmond to see if they would allow you to get an internet line there that you would use to rely to Phatheni (they may even have one already and they would be keen to seeing it upgraded and share the costs with you)

In the long term, and with more community involvement, you may want as Steve says, to go all the way to Pietermaritzsburg. The cost of internet there will be cheaper and over time the investment on the tower needed will be paid off thanks to the difference. Although, I will start in Richmond for now, if you have those long term plans, it would be interesting than when selecting the place in Richmond, you explore how that place would work in terms of arriving at places with fiber optic in Pietermaritzburg. UKZN gets its internet from SANREN, and you can check their points of presence in the fiber here: https://linkmon.sanren.ac.za/nrenmap/ Although a quick look shows me that LoS between Richmond and Pietermaritzsburg is very diificult and that for that you may want to find the spot for your relay elsewhere. Another fiber optic provider you may want to check if you this route is Dark Fiber Africa. Here is their map http://www.dfafrica.co.za/network/coverage/

Hope it helps,

carlos

2 Likes

Sanibona Linda noMdu,

I used to live near Richmond in 2011/12 and even then, there was a well developed WiFi service provider serving surrounding areas; unfortunately I can’t remember their name but probably they are on the wapa map. If they don’t cover your area they might be willing to provide a link from one of their highpoints, to your own network that you will start. In other words, to enter a partnership to help you start.

1 Like

A post was merged into an existing topic: Introductions / Presentaciones / Introduções

Dear Linda and Mvu,

Just read this thread and in my opinion Steve and Carlos has done so well with the information they have provided to get you started with your community network setup. Just a reiteration:

  • Identify your initial network coverage areas (take coordinates) and last mile terminating point (your server room) and keep in mind provisions for future scale.

  • Contact the ISPs within your region for possible last mile connectivity.
    (ISP with fiber point of presence closer to your location who can provide the connectivity either through fiber or radio would be preferable. Consider the cost for setup and bandwidth between ISPs and also network reliability.)

  • Your identified ISP together with you would have to do the last mile connectivity feasibility study to know the best options to use in terms of radio equipment (considering their antenna gains for the transmitting and receiving points), number of towers/poles and height, and cost.

  • Use the radio planning tools to plan your coverage points while considering the type of equipment to use and the line of sight between them.
    Steve and Carlos tools (google earth and ubiquiti link planner) are all free and easy to use tools so feel free to make your choice. Google Earth is great for the determination of the elavtion of your area whiles Ubiquiti allows you to select from their range of equipment with different antenna gains to know an approximate throughput you are likely to get and the noise levels when you deploy the network with those equipment.

After these, you’re ready to get your hands dirty and get your network live!

Best Regards.

Fred Kwadwo Aazore.

1 Like

Thanks everyone for your contributions, these sounds like some great tips for setting up a network in any rural area, not just South Africa.
Much appreciated

1 Like

2 posts were split to a new topic: Community Networks in South Asia

Linda and Mvu,

It looks like you are in good hands with the advice so far!

Some more things: You can use wapa•org•za/coverage/request (relevant to South Africa) to find other networks in your area - there may be more than you think. There are over 300 independent small networks in South Africa - that means a lot of people who might be willing or able to help you. (For those not in South Africa, maybe we can help you to start a similar forum for your country?) zenzeleni•net/resources-2/ page has some resources relevant to South Africa, and also for South Africa you can also use wazimap•co•za to estimate how many people could benefit from a network in your area. (Again, to other people reading this, we can help to adapt a website like this to your country - it will just take a few days of dedicated time by a software developer - and to finding the data from your government.) This website won’t let me post more than 2 links, so I put all these links, and more, here: http://wish.org.za/page/how-to-start.

What is surprising to most people, but very useful to understand, is that even the richest, most powerful person in the world, can not have better internet, than you, or a phone or a laptop that can do things that a cheaper one can’t do. (Why do you think presidents and business leaders use Twitter?) You won’t believe it, but some cheap phones today are better than the most expensive ones - their battery lasts longer, and they are easier to use.

If the richest person in the world can get faster internet, then you can get it, and if you can split the costs with just 100 or maybe even just the right 10 people, then it will be super cheap - so don’t think that you have to settle for a broken connection. If you do it right, you can have a network that is faster than the people in the city, just like the community of Mankosi. Even though they are 2-3 hours’ drive from the nearest connection, and that their connection costs 10 times as much - they still manage to sell uncapped fast internet at R25/month.

Because the power of technology improves so rapidly, don’t settle for old equipment or second hand gear, unless it really is good enough - because often you can get new equipment that is many times better and that can last a lot longer, at a similar price - or maybe there is a hidden cost that you don’t know about yet.

An easy way to fund your network is with an “anchor client” model - if you can coordinate bringing the skills, passion, energy and organization to the table, then you can provide a connection to someone like a hospital, school, NGO or local shop or business person, who can afford to pay for good equipment, and who was going to pay for an expensive satellite or fiber connection just for themselves, and then you convince them to rather buy it from you and then you can provide something better for them. If you are lucky, you might just be able to find someone who can help you to make the sale.

Even some of the cheapest equipment nowadays supports capacity of more than 10 times of what even the most demanding people will need - so if you think of the network as a highway, the first person on your network will pay for 10 lanes, (150 Mbps) and will only need 1 lane (15 Mbps). That means that you can let many other people, including yourself, drive on the extra 9 lanes - and use that to bring costs down for those who don’t have so much money - and there are many innovative ways to do this.

An added benefit is if people from your community are doing the work, then the money that people spend on airtime gets spent inside your community instead of going to places that already have strong economies… or contributing to crazy salaries from big companies like mobile networks, because that money can be much more useful to people from your community.

I found myself in a situation just like you find yourself now, in 2003, 16 years ago as of this writing :smiley: I didn’t for one moment think that I would still be doing similar work today, because I had other plans - but here I am - and there are even more reasons to help people connect to information today than ever before.

A good question to ask yourself now already, no matter how crazy it sounds, is to try to imagine how things will be if- or rather when you have 100 network points, or 1000 - or more, and 10 000 people using them - you may think that there are not that many people in your area, but I challenge you to start counting how many people there are… and compare it to another similar area that already has internet. It might happen sooner than you think. It could also take a lot more time than you think.

How will you feel when you see R 1 000 000 coming into your network bank account every month, and all of that money going out again. What will the other people in your community think? What kinds of threats will your network face? What kind of problems are you going to deal with on a day to day basis at that time?

I learned so much about people, our economy, business, entrepreneurship, tax, regulation, government, “HR”, marketing, and so much more, by building a network - a lot of things I didn’t understand when I started.

There were no “entrepreneurship” schools, no courses on the internet, I didn’t even know what the word “entrepreneur” meant, when I started, I just wanted internet and I couldn’t afford it. So I made up a company name, and went door to door and told people I sell internet for the company, are they interested. When enough people said yes, and I was sure that I will be able to collect enough money when I have connected everybody, I charged them for the connection, and I put it in. I was able to do this because I had already seen all the pieces work, and I had access to a computer on which I could teach myself.

Today I know that entrepreneurship means that if you just start, and work to find a good plan and/or mentor, and focus on the outcome that you want to see, and it is something that people will pay for, and if you can keep at it for a few years, then you will be able to reward yourself - and more important than that - internet can be useful, and can save a lot of time, for almost everyone, and building it, you will empower people to achieve more than they could have dreamed possible. And today you already have access to more help than ever before - too much, perhaps - it can become overwhelming. It can save you years of mistakes if you start right, but it is also important to start, and you can learn a lot by just trying things.

Another good thought experiment is to ask yourself - when you have 10 000 people using your network - how many people will be required to help with problems? What kind of skills will they need? How many days will have been spent on people’s roofs… or driving between towers… How many towers will you have? How much time will it take to redo 100 mistakes? (How much time did it take to do it the first time?) Are there people who have this experience and problem already, right now? Can you speak to them or visit them now? How can you learn from from them so that you can do things right the first time?

From my knowledge and experience, everything comes with hidden costs and problems. If you are faced with a choice of 10 things, then often only 1 or 2 of them are in fact as good as you are brought to believe, while the other things will only hold you back and invisibly build toward someone else’s dream at the expense of those of your community.

There are some concepts that are important for you to understand to empower yourself to make the good decisions - and I am sure that many of that will come up on this forum.

People will not often share stories about their mistakes, but if you can learn to ask the right questions, and look for the right things, you will be able to see the mistakes in what other people have done, and it will be easier for you to have a good chance to get it right the first time.

Be super careful of people who say bad things about others. Soon they will be saying bad things about you too.

There are some mistakes that nobody can teach you to prevent and that is just part of learning… and sometimes we make a problem bigger by just focusing on it - instead of on the solution, which may be something else completely, but there are many problems that you don’t have to make - for example, if you believe everything that someone says, who wants to sell you something, no matter how trustworthy or good they seem, or how many promises they make - if you believe it, without checking up on it yourself, then you might find yourself in a difficult position - and they will not care about you, even sometimes after many years of them seeming supportive, I have noticed this in the small things they do, or in the small empty promises they make - at first you might thing that it doesn’t matter, because it is about small things, but it could be an indication of something you need to be very careful of - some people only show their true face when they finally have the power to get what they want, and sometimes for that they need what you have built up and what they want is to get rid of you. A good strategy to deal with this is to listen more than you speak, so that you can hear who speaks up for you - and can see who will speak up for you when it is needed. To make things more difficult: it is easy to be fooled with technology. The Russians have a saying: “Trust, but verify.”

In my experience it will help if you verify more than trust, but trust is super important in Africa, because often times we only have each other to rely on. I found it interesting to see that the people who are the friendliest, are sometimes the people in countries with the worst governments - and I think it is because in those places people need each other more than they need their government. A similar thing happens the deeper you go into rural areas - usually people become a lot more trusting and friendly. Make sure that you help them to learn about the threats from outside, and how make themselves strong against those - but in a way that respects their peace and happiness.

Usually if everyone is doing something a certain way, it is a good indication that it is the best way. In some rare occasions, almost everybody else can be wrong - and that is when it is most difficult to do the right thing. Usually this is when there is a very new, better way that only a few people have learned about yet. The question is how can you realize this unless you are able to test things for yourself?

With that in mind, if there is the smallest thing that you don’t understand, ask - ask here and in other places - not don’t just ask in one place or person and “hope for the best”, but take your time and ask in as many places as you can - there are many “stupid” questions that I asked on the internet when I started, if you search for it, and I think I can still ask stupid questions every day. Even though I was ashamed to have to ask at the time, or still am, I am proud of having asked every question, because that is how I learned. Today I am smarter and more experienced as a result.

Another rule to keep in mind: The internet is like a magnifying glass - it’s an amplifier. If you put internet in a community hungry for knowledge, it will make them smarter, but if you put it in a community with problems, then it is possible that the problems will only get worse - the internet will not remove problems automatically. Also remember that most problems are just symptoms of something else, which needs to be addressed. For example, the internet can make things more expensive if it attracts more people with a lot of money. We as humans often make the mistake to think the symptom is the problem, when all along there was a much bigger cause that needed a different solution. The internet can also show you things that were invisible before, and that can be overwhelming. (Here is a very good, short test that you can take, that will help you be smarter than most people who have supposedly had access to a lot more than you: Test | Gapminder)

This is why I believe that it is very important to start a discussion in your community about what the internet is, how it works, and how it can help you find solutions - and what changes everyone would like to see in your community. Then, at the same time, it is important to remember that you can’t be everything to everyone, and that you will not be the one to solve everyone’s problems, you will just help to bring another way for everyone to help themselves. You just need to make sure you talk about this early enough, so that other people in your community can start talking and finding the things that will help everyone, so that when they get internet they have the right expectation. Just like money, and roads, the internet is just a tool, or a means to an end - another way to make things happen - it still comes down to what you- and your community needs, and how your community will use the internet to invite the change that you all want to see.

When I wanted to study “AI” in 1996, I could only find 2 universities in the world where I could do it - and nobody in my family could afford the cost of studying at those universities, nobody in my extended family could even afford a plane ticket. Then, in 2012, those very same professors who gave that course at those expensive universities, presented a better course on the internet, for less than R2000. And instead of just teaching just 50 students a year, they taught 200 000 students all at once. I was in a class with 200 000 other students, and it was an incredible experience. It was one of the very first online universities - today almost every university has online courses, and many technology courses are free. People are earning money by doing all kinds of knowledge work, right from where they live, anywhere in the world - for this they need good education - which they can also get over the internet - but again, it might be hard to find.

There are also a lot of scams and bad information on the internet - and it also allows our youth to be contacted and influenced by people who have very different mindsets and values from us. It can be good to learn from each other, but it can also damage communities. I home that this forum can help us share tips on how to avoid the bad things, and amplify the good things.

The internet changed the way that I think, and live. It helped me to understand people and things that I thought were not for me, and it turned me into a global citizen, allowed me to travel to other countries, and to volunteer for projects that I believe in. On the internet, I learned that “freedom can mean that you have to be able to live with being offended” - and that it is better than the other options. It helped me learn to be a lot more tolerant of the things that other people do and say that offends me, and it has shown me reasons why they do it. Many of the most valuable things that helped me the most, I learned through the internet, and a lot of it I learned just by helping to build the internet.

The internet is just a bunch of networks, like the one that you want to build, connected together. Just that, no more, no less. Sure, there are some big and expensive networks, but those are not the only options. It started with universities connecting their networks together, then companies, and then volunteers and neighborhoods, and eventually companies started to specialize in just making connections, but you can still do it in any way that suits you, the key is to take the first step and start learning how, and you have already started! The internet has value because it allows anybody anywhere, to communicate with, share and learn, with anybody anywhere else. (Whether they will hear, or listen, is another question.)

People sharing recipes about beer on the internet has lead to a revolution in small breweries and beer brewing. You can buy a 3D printer for less than R1000, and you can print almost anything for cents. People are sharing plans for spare parts for almost anything. The internet has taken sharing to a new level - and the information we shares empowers us so much that we often don’t even have to charge for the information. Here, the most important thing that I think we can learn, is to distinguish between the information forces us to give power to others, vs the information that gives us power - and to find a balance.

Sometimes it might be worth working for a company for some time, just to build a relationship with them - and to learn from them - but some companies won’t allow this, so be honest with them from the start so that you don’t run into problems.

Be careful empowering people in your community who you will become dependent on but who might not be there anymore when you need them - because sometimes as soon as they have the skills, they can be tempted to go somewhere else and help someone else - of course, this can also be a good thing - but empower enough of the people who have deep roots in the community and who will not leave the community. People who will be there, and close to any problems, who can solve it quickly when they need to.

If you want to be really successful, make sure you build a network that is reliable - that it always works, and that it is never down. That way, there won’t be problems for people using it and there won’t be problems for people to contributing to the costs. A big problem in urban areas - and many rural areas - is that it just takes someone needing the network one or two times when it doesn’t work, for them to lose faith. Then next time even if it is their device that is broken, they will just assume it is your network, and they won’t even bother tell you so you won’t even know that they are having problems. They will then start looking for other options, and even if there are no other options, they will invite them and will switch as soon as they can.

Build the best network you can, and learn from the best. The best is not always the loud person who says how good their network is… it might just be the network of the quietest person. Paying more does not always mean better, sometimes the best advice is free… but sometimes it is also worth paying. Perhaps the best thing you will learn is how to distinguish good advice from all the other advice - something I am still learning.

The good news is that our brains learn automatically - perhaps learning is a symptom of listening to enough information, even if you don’t understand all of it… just expose yourself to as much information as you can, and wait. When you review the same information 2-3 days later, and then 1-2 weeks later, you might find that some more things make sense. Or even if it makes less sense… eventually you will find better questions to ask, and those questions is how you learn. If you can get in the habit of learning by listening and reading with an open mind, like this, you will achieve a lot more in just a few years, that many of us have achieved in decades. If you visit Mankosi, and talk to the old people there, you will see that nobody is too old to learn.

Wow, I wrote a whole essay. But it felt like 5 minutes.

Good luck, reach out early, and often! :rocket:

1 Like

Hi Mentor, I think the older you get the more the writing…:slightly_smiling_face:. Hope you well. Learned the hard way regarding using old equipment or rather second hand gear. Which brings me to a question, what’s a lifespan of a new ubiquity radio eg Powerbeam m5? How can you actually tell if a second hand gear is good enough? Some people say you can use it for three years before it starts deteriorating or rather giving you crap traffic. Does it depend on the usage ? the more traffic it pushes the shorter the lifespan of the radio?
Why I’m asking this, we have sites where we have installed new radios and within 3 years we start experiencing issues, some sites have been running for almost 4 years without giving us issues. I have discarded the thought that other radios could be fake because they all come from the same manufacture unless there’s something I don’t know.
Would be interesting if the was a way to determine the quality of second hand gear prior buying it or using it.

2 Likes

Thanks. It would really help if what brands and models are specified. We all could make a point of reference for future deployments and for performance comparison.

Hi @BONAKUDE - Linda and Mvu,
So many great resources are now listed here (thanks to you all who have posted!).
I know that some “how-to-set-up” questions were also recently addressed at an online meeting held by the Internet Freedom Festival group:
https://internetfreedomfestival.org/wiki/index.php/Community_Updates#July_11

Huge generalization for most equipment:
Old equipment = 30Mbps 1 … 5 years
New equipment = 150Mbps 2 … 10 years

It can of course go faster, but eventually if there is enough antennas and radios, then you will have had to change the settings down to this level for you to have a good connection.

This is based on about 15 years experience with Ubiquiti and Mikrotik and some other equipment… in my experience, most of the devices last between 5 and 10 years, with some of them only lasting a year or 2. Of course it can last longer, but if you count on that you will probably have problems. Most devices on the inside is the same as many others, just different software and plastic and names on the outside.

The biggest thing is how the equipment was treated and installed. If the wires weren’t secured properly, then the network sockets break off inside, or make poor contact. If the radio wires weren’t sealed properly, then the radio short circuits and burns out or becomes very weak. It can take a lot of time to test all of this.

Most of the equipment available today is from the older Wi-Fi (802.11n) or the newer (802.11ac - wave 1 or wave 2). (There is much newer “Wi-Fi 6” but most of the new faster devices today are just using newer features of the ac wave 2) (Your goal is to become smarter than me about this :smiley:)

The M5’s fall under the older ones (802.11n).

For backbone connections, I don’t like to guarantee anything more than 30Mbps and/or 10 concurrent different radio connections on those - even though in some instances you can get a bit more. The remaining lifespan on it may be anywhere from a year to maybe even 5 years or more.

For the newest (ac and ac gen 2 and/or prism) equipment (Litebeam AC Gen2 or Rocket AC Lite), I similarly won’t guarantee more than 150Mbps or 30 concurrent radio connections.

Even though you can get 300Mbps sometimes… And even though the specification says that you can get 433Mbps with one antenna, 867Mbps with 2 antennas, and with MU-MIMO setups up to 6.77Gbps… it should be clear to you that what is possible is rarely achieved - and what is practical is roughly half of the low end of what is possible. Remember that Wi-Fi can’t send and receive at the same time - so if the sync speed is 433Mbps and you do an upload and download at the same time, you will usually get less than 100Mbps up and 100Mbps down - and this diminishes with the number of radios connected. Mikrotik somewhat improves this with NV and NV2 - and Ubiquiti with AirMax - both are “TDMA” solutions that are not part of the Wi-Fi specification, that essentially just means that the time is split into slots and each client gets just a certain number.

Its not about the traffic, it is more about the power supplied to it, and the output power that it was configured at, and the way that it was treated - was it installed properly, sealed properly, cables fastened properly. So if it comes from a company with good quality control on their installations, then maybe 3 years is realistic. If it comes from someone who just gave it to people and said “Install!” then most of it is probably already broken.

Sounds about right!

If you work out the money you saved by using this, taking into account the time spend finding problems, and going out and replacing or fixing it, and if you compare that to the cost of just installing new equipment in the first place, you will often find that the new equipment works out cheaper - and of course lasts longer.

The best is to speak to a lot of people - sometimes there are bad batches of equipment - for example, one factory supplied a bad batch of components and some of the radios, even though they are all “M5” - they have different versions or batch numbers and they all contain the broken components. This is also why it can be useful to track exactly what you are installing and where, so if you find something like this, you can pro-active plan - and have a conversation with the manufacturer.

I hate throwing stuff away - it just feels SO WASTEFUL. But it’s a sad reality in that mass manufacturing is cheaper than mass testing. I like to fix broken gear in my spare time, and doing this I have learned a lot about electronics and engineering - and there are many videos on the internet, and the equipment you need are getting cheaper and cheaper.

But it is time consuming. And there are warehouses and warehouses full of old equipment, because nobody knows how to train enough people to fix it.

In Thailand and Shenzhen I saw (on Youtube) city blocks where there are just people sitting stripping off components for resale, all day long. Maybe in a few years’ time we will have developed better testing tools and methods so more things can be reused better… It’s a hobby of mine to fix old equipment. Maybe we can start a thread about it here?

I think you can set up a test bench, and teach someone a few basic steps - to sort out bad equipment from good. This could be a good learning opportunity, but it could also be that there is a limit on the time people have and that their time will be better spent connecting people in the time that it would’ve taken them to do this… if you can fix and reuse equipment, do it! But work out the real costs, taking into account the time taken, and don’t pay for old equipment. Or at least don’t pay more than you can afford.

I don’t have a great testing workflow, because I haven’t had much time to spend on setting something up - but maybe someone here can share their workflow and steps in a way that is easy for anybody to read and learn from and follow, even if they don’t know much yet…?

What we have been using in the last 2 years for backhaul is Ubiquiti Litebeam AC Gen 2 for links up to 15km ($50 x 2). Ubiquiti Rocket AC Lite + 31db Dual Polarity 5Ghz dish ($100 x 2) for links up to 30km. (They do work maybe up to 7km further than that, but then there really can not be any other wireless noise in the air, and it will be more difficult to get the connection to be stable.) They are all compatible with each other.

These are great for single networks or networks that are very rural and you can comfortably carry 100Mbps on a network like this.

You can also use the Litebeam on the tower (smaller antennas have a wider coverage area), and the bigger antennas for clients that are further away (the bigger the antenna, the narrower the beam.) Near urban areas it is better to use more expensive sector antennas because it allows you to use more channels and to connect more clients to a high site because the radiation pattern is much cleaner - the radiation pattern from the Litebeam is quite messy so if you can afford to use bigger dishes, always do it - but bigger dishes on the tower means you have to put a lot more radios on the tower.

Here is a good guide for the Ubiquiti equipment: https://help.ubnt.com/hc/en-us/articles/205197750

There are some cheap equipment that works, but it only works if you make very few connections. The best value for money equipment at this point seems to still be Mikrotik and Ubiquiti - both allow you to make connections as good- or better than other equipment that costs 5x - 10x as much.

But just because these ones are the best today does not mean that they will always the the best, that is why I keep testing a lot of equipment. I am trying to generate enough revenue so that I can pay for someone to document in detail all the equipment we test so that everybody can contribute and learn from that.

If there are enough enough people and urban development in an area that there is a case to be made to build a shared backbone that can carry 5 or more different independent networks, and in that case it is recommended to put your efforts together so that you can build licensed frequency backbone on 900Mhz, 3Ghz, or 6Ghz - 40Ghz - for which you need to find specialized brands - and then each link will cost maybe $1000 - $2000 - but if you split the costs, it will be same as the slower backbone - but it will give you reliable 1Gbps+ connection to share, without eating up the limited, cheap, 5Ghz space which you will need to connect from your towers directly to buildings, hotspots and busy areas.

Again, be careful to use what they say as your basis for planning - the figures that they give are for lab conditions - if there is no interference and nobody else using any other equipment in the area where you build this. The real world does not look or work like their lab - soon there will be 2 or 3 other networks - and they might think that they know better and that they don’t have to work with you. (There is a lot of value for all of you if you can help them to see the size of the market, and force them to make their calculations so that they can realize that they can’t do all the work themselves, and then to see that it will actually help them a lot to cooperate with you.) Near urban areas with lots of development, like for example in areas around Cape Town today there are more than 100 different companies putting up equipment - and almost nobody coordinates channel use - so I might put up a link on a channel that works, and tomorrow someone else who doesn’t understand Wi-Fi puts up another link on the same channel, and because his antennas are closer together than my antennas, my antennas can only hear his antennas, and my link stops working - then my monitoring system tells my my connection is down, and then I have to log in and find another channel that works better… and every time I have to do this it takes a lot of time, and during that time people can’t use their connection and they become more and more frustrated. You can set the equipment to automatically change the channels when that happens, but sometimes it still results in down time of minutes to hours… and this almost never works in a busy area. This is why it is super important to understand how antennas work, and to plan your antenna positions very carefully - so that they are in areas where they will be unlikely to see too many other antennas. Like in instead of on a tower or on a roof, rather put it in the middle of a wall, or in a corner between 2 walls, if you can.

They will say that you can get 1Gbps on a 50Mhz channel… but maybe your antennas are a bit too far apart, so you only get 500Mbps, and then maybe there are not any 50Mhz channels available any more at some point in time after you installed the equipment, so you can only get a 20Mhz channel, and only 150Mbps. So you might as well have just installed the much cheaper 150Mbps equipment because you can’t get more than that anyways and you wasted your money to buy more expensive equipment - and you also wasted your time. All the equipment, cheap and expensive, just use the same “road” - the spectrum - but specifically, the frequency band. If that road is congested, then nobody can drive on it, no matter how fancy or expensive their car is.

Licensed frequency is like a private road or lane that nobody else is allowed to drive on except you. It is like a bus or taxi lane in the cities and only the bus or taxi is allowed to drive on that road or lane. Where 5Ghz and 2.4Ghz (the license exempt frequencies) is like the normal road - 2.4Ghz only has 3 lanes, and 5Ghz only has 10 lanes - licensed bands have another 30 or so lanes and you have to pay a yearly license fee so that your regulator can afford to police them for you. Although in Africa you have to police them yourself - but usually it is not a problem because the equipment is so expensive that the people who can afford to buy it will also be able to buy the license. The lanes are like “Channels” - there are more channels, like in 2.4Ghz some equipment shows 13 channels, but the way that the settings work is that it is like you have a truck that takes up 4 lanes… so you can only fit in 3 trucks next to each other in the 13 channels, that is why I say 3 lanes. Different networks can not share these lanes so only cars from one network can drive on a lane at the same time, meaning only network traffic from one network - unless you coordinate with other networks so that you all use the same network and equipment that is compatible with each other and then you separate your networks with just settings.

I think it is important to explain things in a way that anybody can understand it by using images that everybody can see - if we use all the technical terms, then some of the people who really need to understand it just thinks it is too difficult and then they don’t even try, when in fact it is a simple concept that anybody can learn if you give them a chance.

1 Like

Thanks for the link, really helps when it comes to deploying the right antennas for different environments.

Hi
Hope I’m not asking this question on the wrong platform but I’m gonna go ahead and post it since I could not find any other relevant category.
We have a network with 3 upstream providers (Internet Service Providers ) ISP that are providing us with IP transit. Everything was working fine for over 3 months but now of late (just over a month) we experiencing problems where some websites can’t be reached and some devices can’t access specific content on the Internet.
Various troubleshooting methodologies have been followed with no luck. Each ISP provisions us with Public IP addresses (5) and a gateway. We use static IP addresses for last mile and we have NAT enabled. We don’t have an ASN and public IP addresses and we use BGP peering for connecting with the upstream providers.
Our core router is a Mikrotik 1036 12G-4s using it’s firewall capabilities and not a separate firewall switch.
Error message we getting when denied access would just be page can’t be found.

Typical example of what we experiencing, a Amazon fire stick would go online and play youtube and other apps online but deny you access to Netflix. However Netflix would work on the same network using a laptop/ mobile device.
An example of a website which is not accessible on all devices would be the yahoo mail.
Kindly advise if you have come across such an issue.

Lol, this is not a “small” one - but just some ideas, I’m sure you will stumble on the cause if you troubleshoot these, at least.

Best I can say is to avoid load balancing, in favor of hash based or failover, until you have the expertise to do it properly - and troubleshoot problems properly. How does your router determine where to route traffic? There are many strategies, and all of them cause different problems… eg. round robin, failover, hash based - it’s just the mechanism by which the router decides which ISP to route which connections to.

Does the same problem still happen if you unplug 2 of the upstreams? Can you identify the broken upstream ISP this way?

Are you using the same public IP address on all upstream hosts? If not, you have to ensure that you have a source routing (or SNAT) rule on your gateway router - so that the traffic will return through the same interface that it was sent from. Many services (such as Netflix) might have security measures to block traffic if it looks like it was “hijacked” - ie. a connection is running from one IP address and then the same connection suddenly continues from another. It sounds like you have an SNAT setup with different outgoing IP’s, but with a local host-based hash - so that the FireTV box is getting its connection to Netflix through another ISP than the other devices. (Even if you load a site like http://checkip.dyndns.org from each device, to check - it might hash those differently from Netflix, for example.)

Also, your main responsibility is establishing end-to-end connectivity - so a good first step is to ensure that the “not found” page is coming from the remote site, and not from one of your local devices - eg. Netflix might have policies in place to block certain devices - the way to deal with that might require more knowledge about their policies and how to work around them with - by routing through another ISP, or using a VPNs and so forth - but as a “network service provider” you’re not obliged to help with that - unless it is actually your network setup that is interfering with the connection.

So you can start by adding firewall logging rules on your exit point - and to confirm that the traffic is actually leaving your network, and returning from the remote network, and also that it’s doing so on the right connections.

Some networks might have explicit or transparent proxies, and they could be misconfigured, or set to only allow traffic to- or from- certain devices. The upstream ISP might have the same - in which case you need to raise the issue with them.

Then, there might also be MTU problems - some routers or devices might be outdated and dropping certain connections because of the packets being too big, and some responses might not be coming through. So a last resort would be to check the path MTU, and lowering the MTU on some of the ports if you see that certain connections are getting broken.

Thanks for the suggested troubleshooting ideas.
The same problem still happens when the other 2 upstreams providers are unplugged which leads me to a conclusion that the problem is not with either of the upstream providers.

It’s definitely our network setup interfering with the connection because the problem suddenly disappears when a mobile network connection is used.

Will look into the firewall logging rules in depth and advise on the outcome. Resolving such issues take time unless you have the expertise off course and the experience, during that period the network usage will drop resulting to cancellation of services.
You might be right regarding the MTU problems, come to think of it the problem actually started post upgrading from using a Mikrotik RB3011 to using a MikrotikCCR 1036. Although it doesn’t make sense because the CCR should be capable of handling higher traffic volumes than the RB3011.
Will troubleshoot further and share the outcomes, hope someone in our Community network has come across the same issues and advise how the issue was resolved.

Thanks again for the advise, “babysitting a network” is no easy job especially when the demand for services increases rapidly. Community networks have a potential to connect masses of communities and this can only mean that our responsibilities as technicians and network providers becomes heavy. Skills development will be a critical component…

1 Like

You’ve tried this with each upstream individually and it does the same on all 3? I know that’s a tedious way to make sure it is your equipment… but at least you can be sure then!

They are based on different hardware, so a different driver is used for each hardware. The problem can be fixed in software without changing the MTU - and might be - so also make sure you are on the latest versions.

Even if not resolved, you can work around it with firewall rules - try pinging with the DF flag set to find the path MTU, then clamp it as per https://forum.mikrotik.com/viewtopic.php?t=130501

I think a key troubleshooting step, is to write down everything simply - and all the permutations. What often happens is that during troubleshooting, things change, or you forget a critical step, then you compeltely miss something basic. I know I have spent hours looking for problems in all the wrong places, when all I did was to forget a . or a , somewhere, or had one digit mistyped in an IP address. Triple check evertyhing in every step and make sure that nothing else changed - eg. I always keep a continuous ping running in the background, to every device that must work for the test I am doing, so that I know if I am being affected by some connectivity issue somehwhere along the specific link. I think just this has saved me hours of troubleshooting.

Had to actually spend the Friday evening at the DC to resolve the issue. After resetting the Mikrotik tried the exercise of testing each upstream provider separately again only to discover the the problem is from the Nap traffic. When the Nap traffic is enabled some of the sites don’t work but when disabled everything works fine.
Will troubleshoot further and share, it might help someone else who’s experiencing the same issues.

Aha! Progress! Friday afternoon/network troubleshooting… :slight_smile:

Hmmm… Also breaking my brain here, I haven’t dealt with anything like this in a year, so I might not be the best person to ask… But I know if you keep trying you will figure it out.

Are you using BGP? Did you give it enough time to catch up with all the routes? (Do your peers have a filter to prevent someone else from using your ASN? Can you access it remotely when your router is down? ) Is your traffic on that connection leaving your gateway router with the right source/return address? If not, it might get filtered at the remote end. Do you have another IP address that you can try? Do you have any ideas of what it could be?

Do try the peering mailing list to ask if anyone else can think of solutions or are having/have had the same or similar issues.