DVDFab Forum - Lack of Proper Regression Testing Prior to Beta Releases


No announcement yet.

Lack of Proper Regression Testing Prior to Beta Releases

  • Filter
  • Time
  • Show
Clear All
new posts

  • DMS v2.0.0.3 Beta Lack of Proper Regression Testing Prior to Beta Releases

    I have been observing over the past year the myriad of posts and responses regarding the beta releases. I personally have not engaged in any of the testing, so I hope that my post is not offensive to those who have. It is clear to me that the DMS team is not conducting proper regression testing prior to releasing beta versions. Each version tries to fix one problem and breaks functions that were working. Clearly the DMS teams needs to take several steps back and figure out how to fix the problems with the scraping and posters without affecting playback and other vital functions. The purpose of regression testing is to make sure that all previous functions are working properly with the new updates. This ensures that the core product is not affected by the updates. This is standard SW/App development practice. Moreover, the fact that the scraping and movie & TV poster functions have not properly worked since they were introduced is clearly a sign that the design or technical functionality is not ready for prime time and needs to go back to the drawing board. First and foremost this is a media server, so the focus should be on making sure that all supported media can be played and played correctly, then focus on file and media management. The initial version allowed the user to control the poster function locally. Give the users back that control until you figure out how to make it work in the cloud.

    Finally, to those of you who have done extensive beta testing, I want to personally thank you for keeping the DMS team honest. If it were not for you, we'd still be on version 1.0.?.?. That being said, I really feel that the DMS team is misusing the beta testers by having the testers figure out what the problems are with each beta release. The DMS teams obviously doesn't understand that these testers are loyal customers who believe in the product and deserve better than the frustration and time ill spent doing the job of the DMS team. My suggestion, for what it is worth, is the DMS teams needs to stop with the endless beta releasing, go away and do not return until you have a finished product that we can use immediately and works the way it suppose to work. This doesn't mean that you completely disappear, but provide timely updates and a clear schedule with reasonable dates of when the product will be ready. If that is not possible, refund our money so we can move on to another product. One last point, the DMS should at least provide the beta testers with some sort of recompense for their time and effort, if nothing else, than out of respect for the work they have put in you behalf of the DMS team.

  • #2
    Not that I am totally disagreeing with your post, I agree with most of it. But I do want to point a few things out. First, as a developer myself, it is not always as straight forward as they fix one thing and break something else. Though it does seem like that at times. In the case of 1.x.x.x to 2.x.x.x fixing 1 thing and breaking another is out of line. The engine is all new, the platform is also new. So what was coded prior and working can be used as a template, it is not carried over code, it is ported. The other problem with alpha testing in a lab is either you use lots of "fake" data or a little bit of real data. ( movies, shows, videos, internal drives, external drives, nas, raid configs etc. ) only so many possibilities can be thought of and reasonably built for testing. This is exactly why beta testing is released to US to play with. 10 of us will most likely spawn 10 different configurations. Also, we all have different collections, naming conventions, path layouts etc etc. And we also are knowledgeable about our own collections so we notice the quirks, what is missing, what matched wrong etc. YES,Thank you all for testing and providing feedback! I admit, I tried a few 1.x versions and simply reverted to the "best" for me version Until 2.x was released. It has much more potential to do what we need. So, I did update to the beta and I am looking forward to the next beta release. I have to admit, some of my early frustration was lack of knowledge in the new version. It still is a pain. A beta user manual would be nice. Granted, a full blown manual is not practical at this point, as the target is still moving. But new features and how they work bullets would be helpful.


    • #3
      Thanks for the response, I really appreciate the feedback. I understand all of your points and do agree with them to a point as well. Hopefully the beta work will get us to a point where we can be, at least 90% plus happy with the product.