Darren Schreiber

No, but our next big project is redoing all phone number management. We are tired of people sending in tickets for buying phone numbers (as I'm sure you are). The issue is that we used to rely on bandwidth.com for their API, and then they changed their API with only 30 days notice, so we had to reprogram for the new API quickly and a lot of it still didn't work right. We're circling back to fix that. The goal will be that you can order from any of our providers who have API support, including number porting, and complete the process yourself. It should be "just as good" as if you had access to the carrier's own upstream portal by the time we're done, with all features exposed except directory assistance. But that is probably 4.1 or 4.2 at this point. Do note that the issue with 4.0 is it was such a massive overhaul, it took over a year to do. But once it's rolled I expect much more frequent rollouts and feature updates because they will be less risky and change less portions of the code.

No, 4.0 has nothing to do with this. There is already a failover feature in the backend that if a phone is de-registered it will trigger use of the call forwarding number programmed onto the line instead. We can probably just add that to the GUI if you want that.

I'll reach out to the folks who integrated and see if I can get more data.

Try it again?

Are you talking about Lync, or Skype?

This one is in debate, mostly because Webphone still needs some work done to it. I'll defer to the product team for announcing Webphone / Operator Console once they're ready to go. Google has made some changes to WebRTC in Chrome which are mucking everything up.

If you've signed a contract recently (last 6-9 months), your costs will stay the same for what you have now and even for a few new apps which will be introduced. Not everything will be free that is introduced in 4.0, though. The biggest item is likely call center lite. Most companies charge something for Call Center / Call Queues so we are debating how we should position that right now. Obviously everyone wants free, but it cost us about $250k to develop all the queue stuff so far, so we have to recoup that somehow. So likely we'll charge something. We're actively learning what other WHOLESALE vendors charge now and how that's marketed retail today. Most of the time it's some base fee for their product plus $ per agent, and/or $ per queue. We'll likely do the same. We will not be open-sourcing call queues for the time being. WebSockets is included and is open to everyone at no cost. We are moving to an app-exchange based model, where each app costs something (either an annual flat license or a usage based model, such as per-user or per-queue). That reflects that we can't possibly maintain a zillion features for a few bucks a month per product offering. It also allows others to put stuff into the same app exchange and charge for their work, too. The goal being you have the most options, and even a few choices for the same feature. The pricing and purchasing will become automated. You go into the app exchange, you look at the apps, you buy (or if they're free, you just turn them on). Simple. Just like every other application exchange / store out there. Check out Atlassian's market place as an example. We're probably closest to that model at this point - open-source core, open-source a bunch of the core/main apps, some apps can be added for a fee as an add-on, pay for hosting if you need that.

We won't be turning off any old servers until we've given a reasonable amount of time for people to move. Don't freak out about the December 2nd date in regards to this particular issue. Let me clarify the dates some more: Phase 1: Upgrade of Hosted Platform to 4.0: December 2nd, 2016 Rollout of new features: Once 4.0 is rolled out and we believe things are working well. Likely ETA is December 2016 or January 2017 (some features may still be considered beta, but will be usable / available) Phase 2: Upgrade of Hardware for Hosted Platform: January 2017 Finalization of all contracts moved to 4.0 version / pricing: February 17th, 2017. Shutdown of ancient servers: January/February 2017, depending on speed of decomission and customer cooperation.

It's open to anyone who is logged in.

No. We did add this to the feature request log. However, considering how many complaints are still coming in with BLF, and considering we're coming out with operator console (which I'd much prefer people use instead of this archaic SIP standard), we're going to go in that order.

Posted more about BLF above.

I've updated the White-Label setting in Monster UI on the Hosted Platform. There may be one more round of changes sometime in January when we upgrade our circuits in Chicago but should be good for now for the 4.0 upgrade.

I've been trying to make time to answer all the BLF complaints and questions posted here, but have not been able. I know this isn't super helpful but: 1) We think there are two different issues occurring 2) We've identified and fixed one of the issues 3) We've added a debugger tool in 4.0 that lets us VERY QUICKLY find any remaining issues in case I'm wrong on the above (including the remaining issue we don't yet understand). All of you will be able to see this and the BLF packets directly from the GUI and we can show you how to use it. It's pretty cool and easy. The main problem with BLF is examples. We have to catch the BLF lights go wonky at the time they go wonky OR know exactly which call it is that triggered the BLF to be wrong. 4.0 lets us do that. Which will literally save all of us time and headaches until everything is 100% perfect. I'll write more on this next week. But yes, this is still our highest priority issue.

I've added a Q&A thread here, please use it as needed. https://helpcenter.2600hz.com/2600hz/topics/q-a-re-upcoming-4-0-upgrade

Hi there,      Yes, some of those servers are old Rackspace servers and they have EOL'd the type of server we are on. We can no longer upgrade their speed or memory :-/      So we are being forced to migrate them. And we're taking the opportunity to just move them out of Rackspace.      We aren't going to turn off any servers without checking the logs to make sure nobody's actually using them. If they are, we'll hunt them down and notify them and help them move off. This is more of an FYI - if you can help us out and stop using those IPs, that'd be awesome - one less person to contact manually.      The 209. and 184. address are still used in a few places, I'm aware of that and having that corrected. If you do see it, please point it out (as you've done here) and I'll get it updated! Thank you as always for the help.

Questions about the 4.0 upgrade? This thread will be an ongoing Q&A. I will edit this top level post as people ask questions so this becomes a collection of questions asked and answers. Q: Why am I required to sign new contracts? A: We have a number of varying contracts that have gone out to clients since 2012. Many of them have not been renewed and contain outdated language, outdated pricing, etc. In some cases we have incidents where, for example, calling 911 without provisioning 911 is not something we can charge a client for, even though we get charged $150 each time you do it, so there is no incentive for such clients to "fix" their 911 provisioning behavior. In other cases, we have issues where contracts include services we don't even offer anymore. This all needs to be cleaned up. More important, we've heard loud and clear from resellers that they want faster responses to our tickets. One way to do that is to roll 4.0 (because we will no longer be in a major release cycle). The other way, however, is to make sure that we can more quickly update terms and conditions electronically when they change. Right now, many contracts require snail mail updates to be sent. We are sunsetting this, which is more inline with most other cloud-based service providers where T&C, Privacy Policy and Legal Updates can be pushed via the portal, email or e-sign. This is more a house-keeping routine than anything else. Q: Are you raising my rates? A: If you've signed with us recently (last 6-9 months), very unlikely. This is more to get your contract's terms inline with the electronic items listed above. If you have a contract that is a year older or more, than there may be some changes (mostly decreases) because we now bill for support separately (and optionally). Many customers are still on "unlimited' support contracts, which we are officially phasing out across the board. Those contracts are likely the ones with the most changes. Each contract varies, as everyone has slightly different ways they sell our product. Thus, we're crafting individual rate sheet updates for everyone. The most important thing is that all rates are actually represented, as many items in older contracts are just wrong at this point. Q: Why was I not given more notice? A: We are allowing two weeks notice for the upgrade, in addition to announcing the upgrade at KazooCon and sending multiple emails prior to KazooCon warning of the upcoming upgrade. In addition, we provided a sandbox login for people to test the update before we rolled it nearly 3 months ago. We would love to provide even more notice, but as we get too close to Christmas and staff start going on holiday, it becomes harder to patch things quickly. This is the best compromise we could come up with. As for contracts, we are targeting 90 days to get everyone up to date so that we can utilize a unified legal system and app roll-out process. We think this is the fairest thing for everyone involved, including us. 90 days allows a lot of time to negotiate if you don't like something, while also giving us a reasonable window for when all these contract updates will be "done" and we can switch to a single notification system for T&C and Privacy Policy updates. Q: Why does this upgrade have anything to do with pricing? A: For customers with contracts signed in 2015 or earlier, some of the pricing and names of products are very outdated. In most cases the prices for most items have gone down, but we have split out some services (like support) that cost us a lot of money to offer. Since we're fixing the terminology in contracts, we figured we should update the pricing at the same time. Q: I would have preferred more than a 90 day window for pricing, why are you giving us this window? See above Q: Why are you performing a disruptive upgrade that I get no immediate benefit from? A: The upgrade to get all the new features requires us to "fork lift" the plumbing to a new operating system (CentOS 7). In addition, we really don't want to introduce new features the same day we are burdened with making sure all the old features still work. It's too much. So, actually, the new features are being rolled out but just not turned on yet. It's a two-phased approach. Once support is satisfied that 4.0 at least covers all existing features without new bugs or issues, we'll continue rolling the new features. Q: When will I benefit from the new features? A: As soon as we're sure 4.0 is stable with existing features, we'll start rolling out the new ones. We'll announce each one as we go and you'll always be able to test them first on Sandbox from now on. Q: What is the start and end time of the maintenance? A: The SaaS / Hosted Platform maintenance window starts at 8pm Eastern (no disruption expected at that time, but we will be shifting call traffic to Chicago or San Jose. Normally this isn't noticeable to most clients but it's POSSIBLE some call quality issues such as pops, hisses or jitter may occur). At 11pm Eastern / 8pm Pacific we will begin what is considered more disruptive maintenance (still hoping no disruption, but there will likely be some). The disruption will likely be some calls failing to complete and/or phones failing to register. Calls should ultimately roll to voicemail if we get everything perfect, but there's a chance we won't, so notifying clients in advance is a good bet. Q: Why are the 209.X.X.X and 184.X.X.X IPs changing? A: Those IPs represent our original Rackspace servers. Rackspace themselves no longer supports this version of their servers, they are considered "v1" and they are trying to get rid of them. As such the pricing is higher, we can't upgrade them anymore (they've hit their max) and they're basically dying. As with the general "housekeeping" theme above, we're going to use this opportunity to get our house in order and retire these servers. We aren't going to just turn off servers without checking the logs to make sure nobody's actually using them. If we see usage still we'll hunt down whoever is using them, notify them and help them move off. This note in our announcement is more of an FYI - if you can help us out and stop using those IPs, that'd be awesome - one less person to contact manually. The 209. and 184. address are still used in a few places, I'm aware of that and having that corrected. If you do see it, please point it out (as you've done here) and I'll get it updated! Thank you as always for the help. Q: Is BLF going to be fixed in this upgrade? A: BLF has three possible issues. One we have been tracking and, thanks to extensive help and debugging reports from Rick Guyton, we've finally figured out last week. Basically, a particular queue that processes BLFs was getting "too full" before the messages were processed (this all happens in miliseconds and is hard to catch without examples). We've basically fixed that issue, with a few recurrences still popping up but I think we've figured out how to track them. Note that this issue is VERY SPECIFIC - if rebooting your phone clears the light, it's NOT this issue.. This issue ALWAYS had the symptom that a light would get stuck in the GUI and rebooting your phone would not work - the light would still be there. You'd have to go into the GUI to "flush" it. Many of you have said "oh yeah, that's the issue I'm having" but in testing, we found this was not true! That a reboot of the phone WOULD fix the issue (which means it's NOT this issue). That said, we are still getting some ancillary reports of lights stuck AFTER a phone is rebooted, but we are researching each of them and they all look to be the same root cause. 4.0 itself has some patches that should help with this, but also the maintenance we're doing along 4.0's rollout should fix this as well. So, yes, this should be resolved in 4.0. Again, to see if the issue you're having with BLF is the one we believe is resolved, you should debug by: 1) Restarting the phone. If the phone comes back up and all lights are working, THIS IS NOT the issue we have resolved. 2) If a restart doesn't work, go into the GUI and clear the light. If this works, then the issue IS resolved with the patches we'll be doing. Please note carefully that for the above to be true you MUST do it in the order of operations I listed - reboot the phone first, and if no-go, then clear it via the GUI. If that COMBINATION works in that ORDER, then the bug we are trying to squash is infact squashed in the future version! (Or so we think!) BUT, onto issue number two... BLF is tough to debug. If I'm wrong on the above (and even if I'm not), the real reason BLF issues have dragged on so long is that they're impossible to catch. We must find the call which caused the light to get stuck. Right now, there's a binary data file we can look in, and it's painfully slow to do, and only employees with access to those servers can do it. So to debug BLF today, we have to have a reseller partner (you) willing to debug with us, the exact call that caused the issue, and a lot of time and luck. This is painful. In 4.0, this is moved to a GUI where we can actually see (and so can you) the last BLF packet we sent to a phone. This should make this problem really fast to debug. The third issue is occasional reports of BLF just stopping working and a reboot of the phone itself fixes the issue. We have traced this one to no end and in every case, it's come back as an internet or firewall issue. Every single case. We still want to find a way to resolve this without having to restart the phone, but we consider this a lower issue since we don't believe our software is actually causing this. Nonetheless, it remains a priority after 4.0 rolls out because people are tired of it. We have some ideas how we can work around crummy internet connections and firewall issues.

Also, welcome to San Francisco :-)

"You'd think they'd have that figured out by now. :)" So ironically, if you're curious, this is a well known issue. The problem isn't the airport - it's environmentalists and local neighbors :-)  https://www.quora.com/Why-does-SFO-only-have-4-runways-Why-didnt-they-build-more-during-one-of-the-b... Bottom line: When pilots can't use "visual approach" and have to use automated systems (during weather, fog, etc.) then they can't use both parallel runways at the same time. So all flights get delayed, and it gets worse as the day goes on. "The main problem with SFO's configuration (2 sets of parallel runways) is that the parallel runways are too close to each other (750ft) to allow for simultaneous landings during inclement weather. The FAA mandates that planes doing a parallel landing at SFO must be able to see each other during the landing, which is impossible during cloudy weather. This effectively cuts airport capacity in half, creating major delays." They tried to fix this in 2000 with expanded runways but residents said no: "This of course was met with massive opposition from environmental groups. Reclaiming land in the Bay destroys large animal habitats and disrupts local ecosystems created by tidal movements. " Then the economy tanked, flights were reduced and people stopped complaining. But now the economy has picked up again, so the cycle is repeating.

Skype keeps their stuff proprietary. Lync is a different story. With Lync it can just integrate with SIP. You treat the Lync device as a SIP endpoint.

I am swamped w/ KazooCon, but you have PERFECTLY said what I was trying to say haha thank you

I mean point them at SaaS customers (hosted) on 2600hz's platform as well. Then everyone can take advantage of it.

Looks like you could attach these to the prod cluster, too?

Do you have a working sample of this you can post? I'd love to see why it's not working. Tuly is reporting it doesn't work with our system and in the logs, I'm seeing the phone not send a BYE to our proxy.

Grandstream used to have an option for OBP (Outbound Proxy) - you can turn that off. That usually fixes it.

Directed call pickup has to be programmed on a key. I don't believe you can use it as part of a BLF key. The BLF key is for call transfers, monitoring another extension and for calling another user...