OpenWISP: An open source network management system

Hi folks!

Some weeks ago I met with Federico Capoano (@nemesis), the lead developer at OpenWISP.org , an Open Source Internet Service Provider solution, to understand how OpenWISP could support Community Networks deployments.

OpenWISP can be used to manage your fleet of routers by keeping them updated, monitoring their health, managing users accounts, controlling allocated bandwidth, having a captive portal… all in a centralized maner.

As part of the conversation, he offered himself to give an introductory talk for us, to reply to a set of your questions about the solution.

In order for him to do that, he asked me to help him in priorize your interests in such a tool.

Could you help me reply to this survey to know what parts of the tool are of your interest?
The idea is to help us prioritize the topics of your interest so we spent most of the time in the things that matter most… and we get the most out of Fedrico’s time.

  • automatic provisioning of devices
  • geographic map and indoor map features
  • monitoring charts and info
  • captive portal
  • network topology chart
  • supported devices
  • deployment walk through

0 voters

If there is any other thing that is not mentioned here, please also let us know in the comments so we can consider it.

3 Likes

I think the OpenWISP talk above would be particularly interesting for @steve @carlos @luandro @eric @troian @spiccinini @Lillian @Irenemisoi @msuantak @michaelgraaf @imlaker @lilichara @sokotnono @chako @jamesgondwe @ventolin @Twahir @leandro @hurricos @infrared @paco @DanaBeltran @dbellomo @brunovianna @alemaree @bruna

4 Likes

@nicopace, certainly interesting. For some of us newbies we might need an overview or a deep dive on network management.

2 Likes

A network management primer is indeed an interesting topic!
This talk would definitely inform a bit about it, as tools like these implement network management activities…

2 Likes

great, thanks; looking forward to it.

1 Like

At Newport Wireless Mesh, we are considering migrating directly to OpenWISP. We have more than one volunteer who works with IAC-based systems and want to make sure our contributions help advance the project.

I’m less interested in supported devices, as ultimately OpenWrt support and OpenWISP node support come hand-in-hand.

On the other hand, I’m more interested (and wish there had been an option) in discussing the work to be done to keep OpenWISP’s firmware distribution as close as possible to mainline OpenWrt. Excerpts from an internal email:

Hey Diane,

OpenWISP is definitely FOSS.

I’ve done a bit of research here in the past, it’s definitely something we should consider:

  • setup is intentionally Ansible-driven

  • it provides a fairly complete management pane

  • Automatic cert management, if you need certs!

My “feeling” on the downside is that it’s always seemed like a walled garden – e.g. using NetJSON to ship […] network configuration. [I want to push for more of OpenWISP to use mainline OpenWrt, as its] base system now has a huge amount of common functionality that would enable high performance while also supporting the hardware we use:

Basically, I’m noting that this project makes a lot of its own choices; some are very much in common with [what we use in OpenWrt], some are not. It’d be nice, though not necessary, to help reconcile those differences in choices.

Just my pipe-dream. I am VERY interested in listening to @nemesis about their project, because we are VERY interested in moving to OpenWISP. Only once we’ve deployed OpenWISP can we really contribute :^)

3 Likes

Automatic bulk communication with subscribers is on my Christmas Wish List.

1 Like

I agree! It’s stupid to have to keep reinventing the wheel. But also we can contribute what we’ve learned and deployed.

1 Like

I believe the talk would definitely clarify many of these things, hope you can join.

1 Like

(new) Newport Mesh volunteer here! As stated by the people above we are very interested in openwisp. I’ve chatted briefly with federico on their gitter channel and worked a few PRs in their dockerized version which is not quite production ready. Very interested in attending a chat!

2 Likes

I would definitely be interested in streaming this talk via ISOC LIVE

3 Likes

Hi all, I am very interested but couldn’t find the details for this talk! Did it happen already, and if so is it recorded? Thanks!!

1 Like

Hi everyone,

thank you for all your feedbak and interest in OpenWISP!

Background info

I want to give you a little bit of background info. The new generation of OpenWISP was born while I had been contributing to Ninux and hanging out with Freifunk for quite many years.
Initially I was implementing this work in a project called Nodeshot (GitHub - ninuxorg/nodeshot: Django based tools to create crowdsourcing web apps.), which was mostly a tool for the community to plan new network nodes and visualize the network map.
Since OpenWISP had traction, more contributors and I was already working full time on it, I decided to focus entirely on OpenWISP from 2016 onwards.
I was only developer working on Nodeshot and since I left the project it hasn’t been maintained by anyone.

OpenWISP Demo recorded at Battlemesh v14

I also want to take this change to let you all know that I presented the latest version for OpenWISP at an event called “Battlemesh” in Rome last week, anyone interested can find the recording here: Battlemesh v14 - Day 3 - YouTube.

It would be great if you could watch this recording to find out if you want to drill down on any specific subject.

Replies to your queries and feedback

Now Let me reply to some of your doubts.

My “feeling” on the downside is that it’s always seemed like a walled garden – e.g. using NetJSON to ship […] network configuration.

NetJSON is an idea and a format that emerged from community networks.
There’s an RFC (NetJSON: data interchange format for networks) and this work was funded by the CONFINE project in which many community networks have actively participated, for more info see:

[I want to push for more of OpenWISP to use mainline OpenWrt, as its] base system now has a huge amount of common functionality that would enable high performance while also supporting the hardware we use:

OpenWISP supports mainline OpenWrt, there’s no custom firmware needed, only two packages:

  • openwisp-config
  • openwisp-monitoring

Both available in the OpenWrt feeds:

Wireguard / unetd rather than OpenVPN (which will seriously suffer on the AR7161 CPUs we have): New WireGuard based OpenWrt VPN implementation: unetd - For Developers - OpenWrt Forum

OpenWISP 22.05 supports Wireguard, VXLAN and VXLAN over Wireguard.
https://openwisp.io/docs/user/wireguard.html

It is possible to use the configuration advanced mode to configure also any other tunneling technology supported by OpenWrt.

The ones that are officially supported right now offer automated provisioning, that means that all the necessary keys and configuration details are automatically generated by the system.

We want to add more VPN and tunneling automations, like Softether and Zerotier, for ZeroTier we were meant to work on it during the Google Summer of Code this year, but the contributor working on it was hired by a big tech company. Without proper funding it’s very hard to retain talent.

Reimplementing the NetJSON configuration parsers (e.g. netjsonconfig/netjsonconfig/backends/openwrt/converters at master · openwisp/netjsonconfig · GitHub) on the client-side using ucode (which has native uci and ubus bindings): GitHub - jow-/ucode: JavaScript-like language with optional templating

Keeping the config logic on the server has many advantages:

Therefore I would rather invest in the direction of an agnostic implementation rather than increasing the coupling with OpenWrt.

We can still reuse some of those tools if needed, but first we have to talk about and define user needs, then we can talk about implementation details.

Using the well-community-supported prometheus data backend in place of the Influxdb one (though not sure if the architecture supports it, InfluxDB is push rather than pull): openwisp-monitoring/openwisp_monitoring/db/backends at master · openwisp/openwisp-monitoring · GitHub

At the moment we support only InfluxDB 1.8. We have to work out support for InfluxDB 2.0 and we have the integration for ElasticSearch almost complete.
A few years ago we had to discard Prometheus as a second option in favour of Elasticsearch because the latter is more widely used.

OpenWISP can do both push and pull, please read here: GitHub - openwisp/openwisp-monitoring at 1.0
Pull has some advantages over push. When the device sends the metrics, if anything goes wrong it can retain those metrics in memory and send them later on, while failed push operations have no way of knowing the state of the device which the system was trying to reach.
OpenWISP has a similar resiliency mechanism on the server API side too, if it fails to write to the timseries DB, it will wait and retry.
This allows the monitoring system to be very resilient to temporary failures, which are very common in VERY busy networks.
We achieved this by using a mix of push and pull. So far we are very happy with the result.

Basically, I’m noting that this project makes a lot of its own choices; some are very much in common with [what we use in OpenWrt], some are not. It’d be nice, though not necessary, to help reconcile those differences in choices.

IMHO, conforming is not always good.
Sometimes developers have to choose what they think it’s best for the problem they’re solving with the available information and resources (time and money) they have.
We’ve been open to improvements and over the last 4 years we’ve amended some bad choices (like extreme modularization which was holding us back) and fixed MANY bugs, this trend won’t stop, so if more people join with feedback and contributions to accelerate this project we can only all benefit from it.

On our side we have to focus what to work on and the decision is to focus on the features our users need to manage their networks.

Automatic bulk communication with subscribers is on my Christmas Wish List.

If you’re talking about the possibility of sending emails to the users,
I had implemented this feature in Nodeshot (I mentioned this in the beginning of the post) for Ninux, something like that could be very well extracted in a separate module and integrated in OpenWISP.
The code is here: nodeshot/nodeshot/community/mailing at master · ninuxorg/nodeshot · GitHub .
There’s also very popular django third party apps for mass mailing which could be integrated too (OpenWISP is based on Django).

I agree! It’s stupid to have to keep reinventing the wheel. But also we can contribute what we’ve learned and deployed.

I dedicated the main topic of my OpenWrt 2017 summit presentation about OpenWISP exactly to this topic:

Hi all, I am very interested but couldn’t find the details for this talk! Did it happen already, and if so is it recorded? Thanks!!

No it didn’t happen yet.
I have to prepare this based on the feedback of this post, we’ll keep you updated!

I hope these replies help.
Thanks again for your feedback!
Federic

3 Likes

Hasn´t happened yet!
Will be announced in this thread and in the #tech-talks space too!

1 Like