Myths/Misconceptions Vs Facts

Discussion in 'KVS' started by KVS Tool, Feb 13, 2011.  |  Print Topic

Thread Status:
Not open for further replies.
  1. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    Myth:

    Fact:
    The KVS Tool is a specialized web browser application, that has been built with frequent flyers in mind.

    Claiming that the KVS Tool is 'screen-scraping' an airline website is like arguing that Microsoft is screen-scraping the same site when an Internet Explorer user accesses it. In both cases, the 'conversation' happens directly between the user's computer and the website in question.

    Internet Explorer presents the retrieved web pages in a human-readable form and so does the KVS Tool. Microsoft does not collect (or have access to) any of the data exchanged and neither do we.

    As tomh009 & WearyBizTrvlr have already mentioned:

     
  2. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    Myth:

    Fact:
    KVS Tool is an actual web browser and meets the technical definition of a "User Agent":
     
  3. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    Myth:

    Fact:
    Both Netscape & Opera web browsers had released paid versions of their web browsers:

     
  4. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    Myth:

    Fact:
    All web browsers get updated from time-to-time, in order to maintain compatibility with the websites that their users access using those browsers.

    As cockpitvisit had already noted:

     
  5. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    Myth:

    Fact:
    There is no "redistribution" of data involved -- as explained above, just like the creators of any other web browser application, we do not have or need the data, that the browser's users exchange directly with the websites (when they browse them).

    As tomh009 had already mentioned:

     
  6. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    <Reserved For Future Use>
     
  7. Tom1024
    Original Member

    Tom1024 Silver Member

    Messages:
    44
    Likes Received:
    40
    Status Points:
    120
    Update: see the more informative Myths vs Facts about the KVS Tool here: http://milepoint.com/forums/posts/126542/

    False. Actual web browsers show web pages as publishers intended - with all branding, links, and logos intact, among other things. You strip out all identifying data from all the websites you scrape, leaving them with the cost of the data but none of the benefits of branding/marketing/etc, this hits the non-airline sites the worst. You get all the revenue, they get stuck with the cost of providing the data.

    Then where is the address bar of your "browser"? If you don't "have or need the data", and the exchange is direct with the underlying websites, then there should be no problem providing the URLs accessed at the top of the tool window, like every other web browser does.

    False. Web/Screen scraping is used to import data from one system into another that it was not originally designed for. IE and other web browsers aren't screen scraping as they show the website as it was originally intended, with all branding, text, logos, links, etc intact. Furthermore, when someone publishes a website they expect it will be shown in full, as designed, by a web browser. There is no reasonable expectation that their website will be abused and scraped for someone else's profit. There is also no reasonable expectation that the user will be unable to know what website they are viewing when they browse to it.
     
    bluesky, TuxTraveller and antirealist like this.
  8. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    <Reserved For Future Use>
     
  9. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    So Lynx is not an actual web browser then? :)

    [​IMG]

    What about Internet Explorer with images & style sheets turned off?

    The KVS Tool comes with a pre-defined set of relevant bookmarks, which are selected via the Method list box.
    It also contains other conventional web browser navigation controls, such as the [Go] & [Stop] buttons.

    In reality, however, the web page will be rendered differently by each particular web browser and the look of the rendered page will depend on the capabilities of such web browsers (and the preferences of their users).
     
    bakedpatato likes this.
  10. Tom1024
    Original Member

    Tom1024 Silver Member

    Messages:
    44
    Likes Received:
    40
    Status Points:
    120
    In both cases, you still see all of the text of the website, and more importantly, you still know what website you're looking at. Both of which are not the case with KVS.
    Great, so what are the URLs of those bookmarks? In any true web browser, you can see the URLs of saved bookmarks.
    No matter how slightly different a web page is rendered by a browser, all the text and content is shown and not hidden on purpose by the web browsers creator.

    Being so defensive about what you are doing just makes it look like you have something to hide. Just fess up, you are screen scraping websites for your own personal profit, at their expense, and without their knowledge.
     
  11. vbroucek
    Original Member

    vbroucek Silver Member

    Messages:
    305
    Likes Received:
    241
    Status Points:
    470
    Seems to me that some people here like Wikipedia as as source of proper definitions etc. Being academic, I do not consider Wikipedia as proper source for anything, but for the sake of this forum, here is the definition of WebScraping from Wikipedia. No matter how much I like KVS (yes, it gives me all I need in much better form than surfing webs, correlating data etc), but IMHO, KVS indeed is web scraping software.
    On the other hand, I would be very careful with insisting that web browser has to have everything that IE, FireFox, Opera and others have. I have written myself purpose built web browsers in order to stop students from browsing inappropriate sites and to give them access oly to what I wanted to give them access to. I am 100% sure that it was not web scraper, but it did not have "address bar" and other things that some people thing are necessary to define something as web browser.
     
  12. Wandering Aramean
    Original Member

    Wandering Aramean Gold Member

    Messages:
    28,201
    Likes Received:
    61,716
    Status Points:
    20,020
    Wow...in 1998. Yeah, I'd definitely use 13 year old information as the basis for defending yourself. Or perhaps you have something current and relevant?

    Didn't think so.

    As for the claim that it is just a "specialize web browser" or "user agent" please explain how that works when rather than rendering individual pages the tool actually requests successive pages based on parameters provided. It is scripted web-browsing where the data is then reformatted and displayed in a manner quite different from that which the original provider published. I'd hardly call that a special web browser but that may just be because I'm playing in the real world.

    And, while you're in here addressing the "myths" about your service, please address these as well:
    • The media coverage you show as a testimonial is, in fact, edited to change the quote provided in support of your product. Fact or fiction?
    • Certain queries are, in fact, processed by your servers rather than a direct query from the client to the data source (in contrast to the claim above that, "There is no 'redistribution' of data involved ...the browser's users exchange directly with the websites (when they browse them)." I believe that the MPM and routing rules queries do, in fact, "phone home" to KVS servers to retrieve the data in question, though it has been a while since I looked at the specifics of the data sources there. Fact or fiction?
    • You purposefully obscure the "public" sources used to retrieve the data. Fact or fiction?
    Also, can you please clarify how you can claim to not be a screen-scraping tool and then cite as a defense of your product another member explaining how your screen-scraping tool has to be updated from time to time? Such inconsistency makes it hard to know what the truth really is.

    Let us also consider the currency conversion sources. Is the data you are using there - certainly you are not just displaying the conversion rates as they are displayed on the underlying source website, right? - being gathered and displayed to the end-user in line with the terms set by the data provider? Most of those free resources are pretty clear about what is and is not a legitimate use of the data they are providing (the KVS "browser" falling squarely in the latter category) so I'm quite curious on this one.
     
    antirealist likes this.
  13. moa999
    Original Member

    moa999 Gold Member

    Messages:
    752
    Likes Received:
    3,193
    Status Points:
    1,870
    I also don't know the relevance of the Netscape and Opera points....
    They sure have a lot of usage now [​IMG].

    I would also argue that Lynx (and yes I used it as my first web browser on a 28.8kbps modem) displays web pages as the designer intended (or now adays partially intended) them to be displayed, much like many websites serve up slightly different versions of pages for different browsers.

    Anyhow I'll go back to see if I can get Trumpet Winsock working so I can browse the web.
     
    arkleseizure likes this.
  14. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    If a user does not like the UI and/or the way a particular browser's rendering engine functions, then they can simply switch to a different browser :).
     
  15. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    These facts remain fully relevant in demonstrating the existence of the non-free web browser concept as-such.

    Nevertheless, here are 2 other browser products that currently exist and are not free:

    The original provider typically publishes in HTML format, which most humans either cannot or prefer not to read:

    [​IMG]

    Therefore, each and every web browser, by definition, must "reformat" and display the data in a "different" format that is understandable by (and pleasant to) the user.

    Fiction -- and has already been addressed by ma91pmh at http://MilePoint.com/forums/threads/kvs-tool-advantages.2431/#post-38870

    cockpitvisit's comment was correct on a conceptual basis, even though (s)he may have used a different term.

    The FOREX data feed actually comes in a CSV format from a Web Service, not a website.
     
  16. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    And the KVS Tool does not meet that definition, because it does not "simulate human exploration of the Web" -- there is nothing to "simulate", as there is always a real human being, who is using the Tool (just like any other web browser), to browse a particular website.

    Likewise, the article states that "Web scraping is the process of automatically collecting Web information". KVS Tool is neither an "automated process", nor does it "collect" any web information -- it simply renders and displays it (again, just like any other browser).

    Precisely.
     
  17. Tom1024
    Original Member

    Tom1024 Silver Member

    Messages:
    44
    Likes Received:
    40
    Status Points:
    120
    Sure, but in that case I'm sure that your students still knew what website they were visiting and showed the website as it was intended to be seen. That is not the case with KVS.

    Great! So then what are the websites that you have "bookmarked" so we can go use them in the browser of our choice? After all either a) the value in KVS is in the secrecy of knowing the websites that are used, and not in the "browser" itself, in which case you won't tell us the URLs. Or b) the value is in the specialized "browser" itself in which case knowing the URLs that are bookmarked won't hurt you at all.

    If you don't reply with a list of the bookmarks then we know it's a) and that all of your wordsmithing and twisting is all BS and you're just hiding from the airlines and website operators and pretending your not. After all, that's why you call United "X1" - to hide from them.
     
    antirealist likes this.
  18. KVS Tool

    KVS Tool Z Representative

    Messages:
    402
    Likes Received:
    245
    Status Points:
    520
    The links to most of the websites are contained in the KVS Tool User Guide, next to the name of each Method, so anyone can use them in a [different] browser of their choice (particularly if they are feeling masochistic :)).
     
  19. Tom1024
    Original Member

    Tom1024 Silver Member

    Messages:
    44
    Likes Received:
    40
    Status Points:
    120
    For the airline websites yes, but not for the availability and fare non-airline websites. What are those links?
     
    antirealist likes this.
  20. vbroucek
    Original Member

    vbroucek Silver Member

    Messages:
    305
    Likes Received:
    241
    Status Points:
    470
    So, are you saying that you do not correlate and collate data from more than one page and/or source before displaying it in UI? I doubt that.
     
    antirealist likes this.
  21. vbroucek
    Original Member

    vbroucek Silver Member

    Messages:
    305
    Likes Received:
    241
    Status Points:
    470
    Yes, of course.
     
  22. ArizonaGuy
    Original Member

    ArizonaGuy Silver Member

    Messages:
    490
    Likes Received:
    603
    Status Points:
    645
    Why is there so much hostility towards KVS? It's a tool that simplifies things, saves a lot of time and can be invaluable in a pinch.
     
    The Winger, KVS Tool and Terry Quinn like this.
  23. IMH
    Original Member

    IMH Silver Member

    Messages:
    188
    Likes Received:
    138
    Status Points:
    395
    Reading through this other current thread will bring you up to speed on the controversy and provide background to the thread we're in now.
     
    antirealist likes this.
  24. TuxTraveller
    Original Member

    TuxTraveller Silver Member

    Messages:
    159
    Likes Received:
    160
    Status Points:
    350
    From a technical point of view I don't see how you can get around the fact that KVS is a screen scraping application.

    The browser argument really doesn't hold, the tool doesn't render the html content for reading by the user in its entirety. It picks out specific parts of the content to reprocess and display in another format. There is nothing wrong with screen scraping, a lot of useful applications use it so not sure why you're so afraid to deny this is what it is?

    I also understand that people are not really against the screen scraping as such, but the potential redistribution. ie charging money for a service provided by another party (ie airlines and other 3rd parties).

    I'm not a lawyer so I won't comment on the legal issues, but as a technical architect with numerous years of experience in the field I'd like to support those that have already voiced concern with trying to label the KVS Tool as anything other than a screen scraping application. It worries me why someone would deny the technical facts in such a case as this.

    I was a user of the KVS Tool in the past, but I must say that all this unwillingness to accept such a simple technical fact and all the defensive paranoia will definitely make me think more than twice about ever using it again.
     
  25. Wandering Aramean
    Original Member

    Wandering Aramean Gold Member

    Messages:
    28,201
    Likes Received:
    61,716
    Status Points:
    20,020
    And I know it to be false. But the spin keeps piling up.
     
    antirealist likes this.
Thread Status:
Not open for further replies.

Share This Page