Introducing Activity Recognition

Activity Recognition on the Dropcam Timeline Activity Recognition on the Dropcam Timeline

Dropcam stands patiently all day, waiting to catch sight of something interesting. To help you sort through hours of footage, we provide timeline markers and alerts when motion activity occurs. However, many customers receive alerts about things they consider to be “false positives,” like shadows, flickering light or trees moving in the wind. Customers might also receive alerts about activities they already expect, such as people entering a room.

Our goal is to make Dropcam smarter so you can receive alerts about the types of activities you care about. That’s why we’re excited to introduce Activity Recognition, a new beta feature that all Cloud Recording (CVR) subscribers can access through our web app.

How does it work?

Dropcam uses software algorithms to recognize motion patterns in your video and organizes them into categories based on patterns it detects most frequently. Each category is assigned a color and when a new activity is recognized, a color-coded motion marker appears on your timeline. You can:

  • Assign names to categories
  • Filter what activities appear on your timeline
  • Customize what alerts you receive by category

You will start to see your first categories after about a day, and a few more each day until Dropcam learns most of the typical activities in your camera’s view. Categories are dynamic and become more refined over time.

The Benefits of Activity Recognition

To get the most out of Activity Recognition, let’s start with some basics. First off, it’s best to keep your camera in one location so Dropcam can learn about all the activities that happen there and create more useful categories over time. It’s also good to maintain a consistent zoom level and position your camera to capture a large area of the room. As more activities are recognized, you’ll be able to:

  • Make alerts more informative: Check out several motion markers that have the same color in your timeline. When you see what pattern is recognized, you can assign a meaningful label and decide what alerts you’d like to receive by category.
  • Reduce false positives: You’ll find that recurring shadows and other lighting effects are learned as a category. There are also activities that happen as one-offs, which may be classified as “Unrecognized.” You can turn off alerts and filter out motion markers for these categories so you don’t receive as many “false positive” notifications.
  • Find the needle in the haystack: Look out for the color-coded motion markers on your timeline that occur infrequently — they could be interesting events you might want to save as a clip!

Let’s geek out for a minute…

There’s a lot of interesting technology behind Activity Recognition. For each activity, moving objects are segmented and a rough estimate of their paths is computed, among other statistics. Our system then uses clustering algorithms to find and group typically occurring variations of these statistics. Complicating matters, consider the scale of learning these activities — our servers currently process upwards of 8 million activities per day. To handle this mass of data the learned models for your camera are updated online, i.e., sequentially, with each new activity. An added bonus of this type of learning is that Activity Recognition gets progressively better over time.

Tell Us What You Think

As you might expect with any beta, Activity Recognition may not work perfectly all the time. Some events will not be recognized when it seems like they should be and some categories may be difficult to interpret.

We’re constantly improving Activity Recognition, so we’d love to hear what you think is useful, useless or confusing. Give it a try and send us your feedback.

Tweet about this on TwitterShare on FacebookShare on Google+Share on LinkedIn

71 thoughts on “Introducing Activity Recognition

  1. I cannot find what “visible” means and what “alert” means on the Activity Recognition page. Only some motions are pictured on the Activity Recognition page.

    Where do I find specific instructions?

    thanks

      1. Hi Tiffany,

        I get a lot of “unrecognized” activity markers as the Dropcam Pro picks up passing headlights outside the patio door. I have unchecked the Alerts box, it remains unchecked, but I continue to get constant alerts for the same issue. Solution?

        1. on of the economical opoitn would be to use a webcam/digicam wireless if cable then care should be taken connected to your PC ,you need a PC for this devices ,then you can monitor from your PC you can livley record,watch the intruders a web cam will hardly cost you 30-40$

    1. My activities at only one color. Blue. It’s been registering over a week with numerous different activities per day. What’s wrong

      1. Hi Tony,

        This is definitely out of the ordinary. Please send us a screenshot to support(at)dropcam(dot)com so we can help investigate this issue. Thanks!

  2. Love the idea of Activity Recognition. I know you said you were working on it, but on an outside view passing cars last about one second whereas people coming to the front door last several seconds.

  3. I had the issue of false positives of one of my 2 Dropcams all summer while I was 2000 miles away. I finally shut it down. It appears this was reaction to leaves on plants on my patio behind the door the camera is facing. The cameras work great, but the alerts suck.

    1. Hi Robert,

      Activity Recognition was introduced to address this very issue that you experienced — false positives. We hope that this new feature improves your Dropcam experience!

  4. Is the new Activity Recognition feature being rolled out slowly? I haven’t gotten it yet and really really want it…

    1. Hi Amy,

      Activity Recognition went live to everyone on October 10th. Please keep in mind that this beta feature is currently only available via our web app. If you have further questions, please feel free to contact our Support Team at support(at)dropcam(dot)com Thanks!

      1. I think it needs to be easier to see how to enable this. I think you need to select SETTINGS –> GENERAL –> DETECT MOTION EVENTS for it to start learning. I had to enable mine myself. It wasn’t already enabled as the post suggests.

  5. I like this feature. I will say at the moment it needs work; it’s not working with the level of intelligence I hope to see it ultimately possess. For example, I have a camera pointing at my street. It seems to color code cars going left and right differently, a good thing. It struggles to accurately portray movement of a tree, however. I also find that the dropcam very rarely picks up pedestrians.

    I know what you’re trying to do with this “AI detection”. You’re trying to one-up the entire concept of regions. With accurate identification of movement and the ability to ignore certain kinds of movement the concept of a region can be negated. I love this in concept, but it’s not there yet. I guess that’s why it’s called beta :)

    1. Hi Matt,

      We appreciate your honest feedback! If you haven’t already, feel free to add your thoughts here. Being able to access all feedback in one place will greatly help us as we work to improve Activity Recognition.

      Thanks for reaching out!

    1. Hi Mark,

      Great question! You can assign names to activities by going to your Settings > Activity > and clicking on Unnamed. Hope this helps!

  6. Is there a way to edit/delete activities? I have one that starts with my dog walking through the room then about 20 seconds later I walked through. I have that labeled “dog” but I really want to split it out into separate activities. I won’t always want to be notified for “dog” activities but I would for “people” activities. Thanks!

    1. Hi Marvin,

      Currently, the only manual modifications you can make with Activity Recognition are renaming an activity and changing it’s visibility on your timeline. Both of which can be done in the Activity tab of your Settings panel.

      We hear you though! Your feedback will greatly help us as we work to improve Activity Recognition. Thank you!

  7. Hey Tiffany,

    Is there any way to reassign categories? Perhaps I’m using the feature wrong, but every time my dog pops up in the frame, the activity gets marked as unrecognized. Is there any way to go in and change that category to my “dog” category or are they stuck in the gray unrecognized category forever?

    1. Hi Adam,

      The “Unrecognized” category is fixed and is comprised of out-of-the-ordinary activities, such as lighting changes across an image that could qualify as detected motion but are otherwise inconsistent with other recognized patterns. Since your dog likely does something different every time, this is an unrecognized activity – for now. We are still improving our algorithms and Activity Recognition will get smarter over time.

      Hope this helps!

      1. (Electronics) From the moment I opeend the box the Logitech Alert 750e Outdoor Master Security System with Night Vision was very easy to understand and had great setup diagrams with color coded ribbons for stress free setup. Where I was mounting the unit 20 feet off the ground I connected everything in the house and tested it to make sure it was working first. Once the camera was mounted in place I simply adjusted it to the area I wanted to monitor and then used electrical tape around the mount joints as they are not as tight as I would have expected and I did not want to have to climb a ladder after every wind storm to adjust. Setup was very quick after installing the mounting bracket.It should be noted the the ribbon (power) cable that connects to the outdoor camera is so flat and thin that you can bring it in through a window without compromising the window seal when the window is fully closed. This is a huge plus as I did not need to drill a hole for it.I set up email alerts and also I highly recommend if you have some blowing tree branches in view as I do that you set up particular areas for movement detections. If you do this and set the motion rate for recording you will capture only scenes of interest which makes it so much easier when reviewing recordings.The master system comes with the network hardware and the invaluable command center. This is extremely easy to navigate. You can zoom, pan and tilt effortlessly. I am thinking of adding this software for remote viewing as well.The night vision is suprisinging very very good in low to know light. I am extremely impressed with the night vision and have to say that it is well worth the extra money.The only drawback I can see to this system is that if you enable and use the audio component which comes built in and has great range, picks up way to much wind. I would like to see an add on for wind noise reduction. This would not keep me from highly recommending this product. I did a lot and I mean a lot of research (hands on and reading) before making my decision to purchase this camera and I am very glad I did.I highly recommend this product! Picture is extremely clear not washy like others. The audio is good enough to capture voices and sounds with in the entire viewing areas and sometimes beyond.I was not able to figure out how to upload a few pictures with this review otherwise I would have.Thank you Logitech for a superb and well built system that seems to have it all.

  8. Im trying to figure out what makes the different colors on activity recognition, is it by activity in a quadrant of the picture? Is it based upon the location where the activity takes place? And how is that divided, or is it something else? Thanks

    1. Hi Michael,

      Dropcam uses software algorithms to recognize motion patterns in your video and organizes them into categories based on patterns it detects most frequently. Each category is then assigned a color and when a new activity is recognized, a color-coded motion marker appears on your timeline.

      Hope this helps!

  9. Center field of my camera is sacrificed while periphery is where all of the alerts on activities are coming from. Center view, low light never alerts.

  10. I am not getting anything on my events timeline, I haven’t for over a week. I bought these to track activities and I have better things to do than click through the timeline minute by minute. Very close to tossing and buying a more reliable product. I think if there are going to be beta releases, they should be voluntary. I don’t have time to test your products for you.

    1. Hi Bonnie,

      Sorry for this inconvenience. Some customers have reported that their CVR timeline no longer displays events while using the web app, but may appear while using a mobile app. We are currently investigating this issue.

      If you are a Dropcam Original or Echo owner, please contact our customer support for assistance. You can contact us at support(at)dropcam(dot)com or by calling 1-877-345-DROP (3767) Mon-Sat, 6am-6pm PT.

      Thank you for reaching out!

    1. Hi Amy,

      You should be able to edit the names of your activities by going to Settings > Activity and then clicking on the title. If this doesn’t work, please feel free to contact Dropcam Support at support(at)dropcam(dot)com

      Thanks for reaching out!

  11. Is there any plan to implement segmenting off the entire view to only a portion where you want activity recognition to occur ? I had older alarm.com cameras that had this feature for quite some time and it was fairly reliable (I know it wasn’t your algorithmic recognition) I also have a camera pointed at a front door and while Im trying to capture people, it hits on cars and bushes moving etc. (and I cant set a new name/alert to the cars, as it cancels out the people- it would be better if I could manually set names and color codes to differentiate good from bad motion-thereby teaching the algorithm) If I had the ability to segment off those portions of the viewable area, the motion alerts would be tremendously more useful (along with the algorithm) Motion alerts are a must have for me (and it appears many others) Thanks- great cameras.

    1. Hi Pete,

      We haven’t announced plans for this, but we definitely appreciate your feedback! We will be sure to keep you posted as we continue to improve Activity Recognition. Thanks for reaching out!

  12. So, one of my events was mis-categorized. Is there a way to change the category for events? An extension of that feature could allow a user to teach dropcam about the types of events that are most important to them.

  13. Actually this is a question. I just got my dropcam pro. I have it monitoring my front door. I love it! It works great on my computer and iphone. I was just wondering if anyone knows a way to connect the camera to my Direct TV? Direct TV is connected to my wi-fi. I can see who is calling me on my phone. Just wondering if there is a way to connect to the genie box.
    Thank you
    Dan

    1. Not familiar with directv– however, if you have Airplay, you can sling your dropcam video from your ipad, iphone or macbook to your television.

  14. Wow, great product. I spent several days dicking around modifying used android phones to use as cameras in my store after a security consultant quoted me a price an order of magnitude higher than dropcam.

    Activity recognition is a great idea, but I’m having a hard time figuring out what actions dropcam is using to create categories. Scrolling through the timeline looking for bars of a specific color on the timeline is time consuming, and requires repeatedly switching between 24hour and 5 minute scales.

    It would be great to be able to click the thumbnail and see a clickable list, maybe on a sidebar, of the events in that category so I could go directly to those times. This would also help colorblind people navigate the interface.

    Also, because activity recognition is such a new concept. A little more documentation might help. I spent quite a bit of time looking for a view that included the colored dots and labels shown in the introducing-activity-recognition graphic before I noticed the colored lines under the thumbnails in settings>activity and figured out that I needed to find bars of the same color on the timeline.

  15. I am unable to rename any activity or event from the camera settings menu on the web version. I can highlight the default label (ex: Unknown) but when I attempt to change or rename the label that action is not accepted. Could you please advise as to why such is the case– when your information indicates that I am supposedly permitted to make such changes.

  16. I want to make sure I understand something correctly.

    Let’s say a cloud moves and sunlight becomes stronger through a window, and that additional light creates a purple marker on my timeline and I also receive an alert. Then, a little later, someone moves through the edges of the camera’s view, and that movement is also represented by a purple marker and I receive an alert. (This is a real scenario on my timeline, by the way.)

    If I tell the camera to ignore purple markers, will it allow people to move through the camera’s view without creating a marker on my timeline or resulting in an alert? I wouldn’t want that to happen.

    I guess I’m not understanding why changes in light will result in the same color marker as an actual person moving through the room.

    Thanks.

  17. Because they use so much bandwidth, I’m using 2 Dropcams for security only at night, under the roof in a screened in patio. They will detect movement (usually animal) through the screen, but the entire picture is in infrared, so there’s little detail. Am I correct in assuming that activity recognition will be nil, since the algorithm won’t be able to determine what’s moving?

    Also, are you planning on letting me view the CVR video on an iPad/iPhone in the near future, so when I get an alert away from home I can see what’s going on? That would be immensely helpful.

    1. Hey Gary,

      You can always get a fullscreen view of that particular Activity by viewing it on your timeline! Simply click on the corresponding color and watch it as it happens through the viewer.

      Hope this helps!

  18. The new additions in terms of being able to name/color the activity, etc., are great but boy, I really really wish you’d improve detection/recognition (even if unrecognized) of movement when the scenery is backlit. Situation: Front patio, DropCam angled down from ceiling. DropCam will detect movement outside of window in the background (filling only 20% of the “scene”) but substantial/major activity/movement in the foreground (80% of the “scene”) isn’t even detected. The 80% foreground is darker, slightly backlit (scene outside window is bright), but it baffles me that DropCam can’t recognize any activity in this area. I agree, zoning is key — if not, I don’t understand why contrast detection can’t be improved (because even waving in front of the window isn’t detected). As far as I can tell, DropCam requires full frontal lighting of the scenery for it to detect activity. This needs to be specified on the website. I do not agree that the problem is limited to “outdoor scenery.” I’ve moved the DropCam relentlessly and patiently in my indoor patio — angling and blocking as many windows as possible. I’ve purchased multiple DropCam Pros and CVR subscriptions but this is a deal breaker. Will have to hope that CANARY activity recognition algorithm is “ready for prime time” and not in perpetual ‘beta.’

  19. Hi,
    I activated CVR on a DropCam that looks out on the approach to our house. It’s been recording for about a week and still says that it’s learning activities (which may take up to 24 hours). Seems like I have a slow learner.
    Even though it is looking through double-pane glass, it has no problem reacting to tree movement, the occasional bird, and pedestrians. How can I get it to stop learning and let me differentiate the activities and set appropriate alerts?
    And while I’m at it, an exterior DropCam would be even better. Gotta be a big market.

  20. Although I believe is a cool function, it frustrated me that was presented (by Dropcam’s Support), as a solution for the problems I’m having with the camera (Request #153929).
    Ok, i get it. It’s Business and you guys need to sell, and selling a monthly service is a great way to make more money (better margins over hardware?).
    But seriously, I believe the flaws of “unrecognized activity alerts” for the non-subscribers like me, need to be fixed in your product!
    My 2 cents!

    Alex

    1. Hey Alex,

      Thank you for your feedback. I will personally take a look at your support ticket and answer you as soon as possible.

      Thank you!

  21. I recently installed 2 DropCam Pros. I love the ease of use and remote discussions with my adult daughter. You see, not only are my DropCams a home security tool but it also serves as a way for adult children to visually see and talk to elder parents. Now, one camera has become so accustomed to my daily activities that it never sends alerts to my sounds and movements. The 2nd camera remains sensitive. Should I leave the house for vacation, I would like the less sensitive camera to “unlearn” activities. Is there a “reset learned activists option” or must I physically exchange the two cameras so that both cameras react to motion and sound as when they were initially installed?

  22. I have moved my camera and would like to delete the old activities it recognized from the old placement . Is there a way to delete activities?

  23. I have four cameras, three with CVR. One camera has not recorded any events for four days. I know that nothing has moved for four days. Lights go on and off, outside light changes some, but until about a week ago, I could see a few recorded “Unrecognized” events per day. Is this an example of your algorithm. I also use an iPad and an Android to view events, and there is no difference viewing events on any for this one location. I would like my events back.

  24. Dropcam is great- so first off thank you! Activity recognition I would like to shut off… can I do that. I don’t want to disable motion detection all together, just want to have one category for all motion. Also, when it try to turn off alerts for Unrecognized Activity, it does not hold the OFF switch. When I go back into Unrecognized Activity on my iOS it is back on. tks vm

  25. I just want it to email me when anybody enters my vacation house. There should be no other activity, but I have to suppress the false positives. Can it learn to detect the occasional….maybe once a month …visitor (worker, burglar). I don’t care about it learning my activities, because the camera is off when I’m there. In other words, I need a simple security camera for a vacation house.

  26. Hi, I do think this is an excellent blog. I stumblesupon it ;) I’m going to
    return yet again since i have bookmarked it. Money and freedom
    is the greatest waay to change, may youu bbe rich and continue to help other people.

    My blog; lose weight healthy (Clifton)

  27. Herbal vaporizer is the best choice of every health conscious person now a day.
    Wrap the solid herbs residue in a wash cloth and tie with a ribbon or rubber band.
    ‘ Bolster your immune system during cold and flu season with vitamins, herbs and
    good nutrition.

  28. Fantastic beat! I’m going to newbie even as people amend your web site, exactly how may we sign up for the website web site? Your bank account assisted us a appropriate bargain. I actually have . homesitebeen touch comfortable of your your own transmitted made available vibrant crystal clear idea

  29. Hi, Dropcam is a great product, and activity recognition is a fantastic function. I would like to know how Dropcam detect the activity. Does the software algorithms need to get the content of the images?

  30. We just installed the dropcam. It is not picking up all activity. For instance I unlocked my office door, came into the reception area (which is where the camera is, and walked through into my office. Dropcam never sent me an alert. This is the third or fourth time this has happened this week. What could the issue be?

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>