How to configure one-to-one relationships with Entity Framework and Code First

Configure One-to-One Relationship - this site is focused on EF in general, but also has this specific article.

Must have Graphics Tools

Below is a short list of some really great tools for editing or creating graphics, editing photos, etc.

The Gimp - It is the cross platform FREE and Open Source alternative to  Photoshop.

Paint.NET - This is my long time favorite because it is easier to install and use than Gimp, opens faster, and has most the features I need.

Pixlr - Has most of the important functionality of The Gimp, but runs in any browser that supports flash.

Please suggest any other graphics apps that should be included here.

Map Windows Share to a Drive Letter using batch file / command line and prompt for password

The line of code can be pasted on a command prompt in Windows 2000 and newer or pasted into a text file with a .bat extension. It will prompt the user for a password to the Windows share. The username is hard coded

@ECHO OFF
rem net use /del V:
SET /P %password%=Password:
net use v: \\Wtjwh175\c$\dev\dropbox /u:"yourDomain\yourUsername" %password%

Technically, you will get an error saying "The local device is already in use.", if you run it twice. If you don't want that to happen you can uncomment line 2 (change rem net use /del V: to net use /del V:). This will unmap the drive and then map it again each time the batch file is executed.

Passing additional view data to a Edit to make sharing a EditorTemplate on an Edit and Create view.

Imagine you have a Editor in Shared\EditorTemplates\Person.cs for an object (Person in this example) and two views for an Order entity that is showing edit fields for Order obeject and the Person object. The two EditorTemplates are Person\Edit.cshtml and Person\Create.cshtml. To use the Person EditorTemplate in the Order Edit AND Create view we need to make sure we always pass a value for the Person.OrderID. If we don't do this, the problem is that when we use the editor in the context of the Order\Create.cshtml View the Model is null and validation for the hidden field will have a value of null. When it is on Order\Edit.cshtml the model will not be null and we will have a value for model (a Person object) and thus have the value Model.OrderID. It is really the Create that requires this work around, but we want to be able to use the EditorTemplate even when it is on a Create view. Below is the code needed

Code located in the Order\Edit.cshtml View
model in this context is the Order object
NOTE: We ARE passing MyOrderID value here since we do need the editor to know and keep the value.

@Html.EditorFor(model => model.Person, new { MyOrderID = Model.OrderID })

NOTE: When we set MyOrderID, it is scoped to the EditorTemplate that we are passing the value to. However, if we put the value in the ViewBag or ViewData directly (at the top or Order\Edit.cshtml for example) then it is available to all Views and EditorTemplates in this context.

Code located in the Order\Create.cshtml View
model in this context is the Order object
NOTE: we are not passing the MyOrderID value here since we just need any value (-1 was arbitrarily chosen to be safe) and will be assigned by the Entity Framework when it is saved to the database. Most importantly it will pass the validation and the model will be valid until then.

@Html.EditorFor(model => model.Person)
Code in the partial view ( in EditorTemplates folder makes it a template) for the Person object.
model in this context is the Person object
NOTE: If no value is passed using ViewBag.MyOrderID then we use -1, otherwise we use the passed value (usually from the Order\Edit.cshtml View)

@Html.HiddenFor(model => model.OrderID, new { Value = ViewBag.MyOrderID ?? -1 })


You can also use a strongly-typed ViewModel and pass all data to it instead of using the ViewBag.

Configure Code First migrations to use Singular Table Names

If you are using Code First Migration with Entity Framework you will probably have noticed that the table names it generates are all plural. Personally, I dislike this because all the contstraints including foreign keys are very confusing to read when they are plurarlized. Before you do an update-database for the first time be sure to override a method (OnModelCreating()) on your class (typicaly ApplicaionDbContext) that inherits from IdentityDbContext. The default class is called ApplicationDbContext.



public class ApplicationDbContext : IdentityDbContext
    {
        public ApplicationDbContext()
            : base("DefaultConnection", throwIfV1Schema: false)
        {
        }

        public static ApplicationDbContext Create()
        {
            return new ApplicationDbContext();
        }

        public DbSet People{ get; set; }
      

        protected override void OnModelCreating(DbModelBuilder modelBuilder)
        {
            // make the table names singular instead of plural             //when code-first generates the table names

            modelBuilder.Conventions.Remove();
            base.OnModelCreating(modelBuilder);
        }

    }

Review of some products for working with MS Word documents using .NET / C#

I didn't want to use MS Automation because it is slow and not server friendly, so I am not covering that here. The solutions below do NOT require MS Word to be installed and they are server friendly.

Open XML Word Processing SDK
This is my first choice for completely free solutions because is relatively easy for basic things and it will likely always be supported by Microsoft. This is the Microsoft SDK (released as Open Source as of version 2.5) for creating and editing MS Word documents in DOCX format. It takes a bit to get used since it mirrors the internal DOCX format. The advantage is that pretty much everything you need to be done here. The disadvantage is that it may take more code than using other packages. The link above has sample code on how to do many common tasks. Here is a good place to start to familiarize yourself with the file formats, what resources are available, etc. To understand the DOCX file format, check this out. The DOCX format is much simpler than the XLSX format. I recommend giving it a try. The learning curve is pretty small for the DOCX file format and there is a lot of examples available. In the end it isn't so different than NPOI (see section below) for DOCX. Here is a list of how to do's.

It is unofficially distributed on NuGet as DocumentFormat.OpenXml, but it is unofficially distributed by someone other than Microsoft (even though it shows the author as Microsoft). If you want the official binaries they can be downloaded here. If you want the source code you can get it on GitHub. This video shows how to build the source code if you want to go that route. If you want to use 2.6 it has been released on NuGet as OpenXMLSDK-MOT. This is the same as official binaries and source code in GitHub (from what I can tell). All options install an assembly called DocumentFormat.OpenXml and if done through NuGet it is added a reference to your project.

If you want to validate that the document you create is valid, check out this code example.

Here is an example of how to do a search and replace in a DOCX file.

Here is the link to the Open Xml Developer website.

There are also tons of example on the Open Xml Power Tools project on GitHub and from NuGet.

Open-Xml-Power-Tools - DocumentAssembler
This is an Open Source module in Open Xml Power Tools that allows you to create templates using .DOCX files as templates, a xml data source, and generating a well formatted DOCX file with the data merged in to the template. The template allows such things as tables, conditions, etc using XPATH-like syntax. Here is a video on how it works, but not a step by step tutorial. Here is a getting started video / tutorial that walks you through using the product. It is important to watch the ending where he talks using <# #> instead of Content Controls in Word as the placeholders. It works similar to most reporting tools, but using MS Word as report definition (template) file and the output being a MS Word document as well. You can download the entire Open-Xml-Power-Tools suite that includes DocumentAssembler from here. It is not meant to be used to create DOCX documents from scratch. It always uses a template file to generate new DOCX files. However, Open-Xml-Power-Tools suite does have the ability to help work with DOCX files. NOTE: It is built on top of (requires it) the Open XML Word Processing  SDK (above).

DOCX
If you really don't want to learn anything about the DOCX format and want a more inuitive way to interact with the DOCX files then this may be a good option if you are okay with alpha software. A simple Open Source project available from codeplex.com or NuGet. It let's you interact with the Word document in an intuitive manor without understanding how Word documents internally work. The examples on his blog are good and good examples in source code. It is easy to use. It is still alpha, but has been around since 2009. It has growing popularity.

Free Spire.Doc
This is a free version with limitations for a professional package called Spire.Doc. It is quite powerful and also does conversions to PDF and many other formats. It does mail merges, etc. Is is nice for small projects that are below the limitations. The API is well thought out and works nicely.

NPOI
This is the .NET implementation of the popular POI Java Project. It is Open Source and totally free and also does MS Excel. The API is a bit low level at time, but works well. I recommend downloaded from github to work with examples. Also, you can install binaries via NuGet. The documentation and examples for XDOC are not really there yet though. Look for XWPF if you want to use DOCX file format. The problem is that it is not the exact same API as the POI Java Project, but it is similar. It is also not as mature and is thus missing some features. It is a bit more abstract that just using the Microsoft packages, but still quite a bit of DOCX format knowledge is needed. In general I found the experience a bit frustrating because of how the project is organized and the poor documentation and very few DOCX examples.

Resetting migrations when using Code First Migrations in MVC

1. Delete all the migrations folder in your project except Configuration.cs
2. Move Configuration.cs to some place safe and remove from the migrations folder
3. Delete the records in the __MigrationHistory table in your database
4. Then run the following command in the Package Manager Console:
   >Enable-Migrations -EnableAutomaticMigrations -Force
   NOTE: This will recreate your Configuration.cs
5. Replace the new Configuration.cs with the Configuration.cs that we moved to a safe place earlier
6. Then run the following command in the Package Manager Console:
   >Add-Migration Reset
   NOTE: The name is not important
7. This will create a file in the migrations folder.
8. Open the new migration file and comment out all the code in the Up() method such that the body of the method is just commented out code.
9. Save and Build the project
10. Then run the following command in the Package Manager Console:
    >Update-Database
11. Uncomment the Up() method.
12. Save and Build the project.
    
    
    
    

Essential MVC Tools

Below is a list of tools, plug-ins, packages, etc that you should really consider using if you are doing development for ASP.NET MVC.

Sidewaffle - custom templates. Can consume them from library or create own

Glimpse - amazing view into all things. particularly performance. Scott Hanselman has a good review of it.

ZenCoding - very cool html coding that greatly reduces typing of html tags

Web Essentials - extends Visual Studio, including design time edit from browser

Less - gives variables, etc to CSS like files that we can generate css files from automatically

WURFL - on nuget for reliable device detection (mobile, table, google glasses, etc)

Automapper - Copying data automatically from object to another similar object








d
dd

Good Articles on MVC and .NET

Getting Started

Implementing Basic CRUD Functionality with the Entity Framework in ASP.NET MVC Application - Good step by step tutorial for MVC 5 (not MVC 6). It also shows how to use TryUpdateModel to specify what can be bound to each object. This is a bit cleaner and more object specific, but does require a bit of code.

Binding 

ASP.NET MVC - The Features and Foibles of ASP.NET MVC Model Binding - a great post that gives an in-depth explanation of  how the binder works and how it can be extended.

Prefixing Input Elements Of Partial Views With ASP.NET MVC - Explains how to make a partial view generate html with references that the Binder can understand properly. Also shows a generic method for passing the prefix to the Partial View. It doesn't say it, but creating Edit Templates and EditorFor() instead of Partial Views will also solve this problem.

Model Binding To A List - Explains how to set the Name html form property so that the binder will create the collection.

Mass Assignment / Over-posting

6 Ways To Avoid Mass Assignment in ASP.NET MVC - If you use the Include or Exclude parameters with the Bind attribute it doesn't seem so say it anywhere, but the names of the fields are the same as what show up in Request.Form. So, things like Person.Address.Name, Person.Address.ID, and Person.Address would all need to be added to the Include parameter in order for fields bound to related objects to be allowed through the Include() list and be added to the Request.Form colletion.

Sharing Create / Edit Screens

 ASP.NET MVC - using the same form to both create and edit - forum on how this could be implemented

View Model

How to Use ViewModel with ASP.NET MVC - shows how to implement the repository pattern, how to organize your project, and how to use a View Model.

Videos

Building Applications in ASP.NET MVC 4  - very in depth video on how to build MVC applications. Most of it still applies to MVC 6. It has details on certain topics that are not covered in the MVC 5 version of the video.

Building Applications in ASP.NET MVC 5 - very good video and in depth video on how to build MVC applications.

Best Practices

Best Practices for ASP.NET MVC  - this is a bit old, from 2010, but still has some good advise.
 

Dependency Injection

Dependency Injection and Unit Of Work using Castle Windsor and NHibernate - good example of how to use DI and UOW in MVC application. It shows NHibernate, but it can be used for Entity Framework. It also shows how to use in the context of the different layers of an application.

Castle Windsor Tutorial 1 - I highly recommend reviewing this tutorial. It shows how to create a Castle Windsor container application. It is complex enough to see how a whole application can be done with only calling the container 3 times. It also found it useful to modify the code such that it does NOT use IoC (i.e. not using Castle Windsor). This involves instantiating objects by hand. Then a line at a time, I removed the code I added to hardcode the creation of an object and added the appropriate line in the Installer for that object. Run the application between changes to see how the container actually instantiates the objects automatically once they are registered (in the installer).

Castle Windsor Tutorial 2 - In the case where you do need to create your own instances of an object and still use IoC, you should use TypedFactoryFacility.

Krzysztof Koźmic on software - talks about IoC concepts in depth.

What's New

Top 10 Changes in ASP.NET 5 and MVC 6

Entity Framework

Configuring Relationships with the Fluent API i.e. configuring Cascade delete and one-to-one relationships.

Unit Testing

Testing Entity Framework with a MOQ - step by step instructions on how to test the EF6+ using MOQ. I am using the latest EF6 and did NOT have to change the class the inherits from DbContext such that the DbSets are virtual because they are already that way in the T4 templates.Also, if you need to access the .Set method of the DbContext then you will need to tell the mock what it should be returning using something like: mockContext.Setup(m => m.Set()).Returns(mockSet.Object); See here for more details.

Attributes for MSTesting - includes samples of attributes for setup and cleanup methods that apply to tests, classes, assemblies, etc depending on the scope you need. A class can be created that has assembly specific setup and cleanup and doesn't need to have any tests actually in the class itself. Teh class does need to be marked as TestClass() though.

Unit Testing Good Patterns #3 - Know Your Moq Argument Matchers - this is an excellent read to understand how to use the It and Verify classes.

Keeping up

Code Magazine

Entity Framework Team Blog

Using CDN (Content Delivery Networks) with ASP.NET MVC

CDN (Content Delivery Networks) can be a great way to speed up your ASP.NET MVC application for the following reasons:

• Files are downloaded from the best geographical location
• CDN networks are trusted and built for high availability and high usage
• The browser can download files from several servers at the same time.
• CDN networks can use caching to further improve performance

With ASP.NET MVC  we can specify a CDN to use AND also a fallback url in the event that the CDN is not available for some reason.

In BundleConfig.cs you can specify a CDN (Content Delivery Networks) url and a fallback url (on web app) if the CDN fails for some reason. The failure is determined on the browser side by checking a javascript expression specified by CdnFallbackExpression.

Here is an example of what can be done for the ~/bundles/query that is set by default on a new project.

            // use CDN if it is available.
            // NOTE: Won't show on local machine unless turn debugging off in web.config
            bundles.UseCdn = true;

            var bundle = new ScriptBundle("~/bundles/jquery",
                @"//cdnjs.cloudflare.com/ajax/libs/jquery/1.10.2/jquery.min.js");
            bundle.CdnFallbackExpression = "window.jquery";
            bundle.Include("~/Scripts/jquery-{version}.js");
            bundles.Add(bundle);
 
Be sure to set the bundles.UseCdn = true. Also, if you are running your project under debug (set in the web.config) then you won't get the bunding, minimizing, optimizations, etc, but you would in production. To force these in development, set debugging to false in the web.config.

Here is a list of CdnFallbackExpression values for common CDNs.

## Library ##      ## Expression ##
jQuery             window.jQuery  
jQuery.UI          window.jQuery.ui  
Modernizr          window.Modernizr  
Bootstrap          $.fn.modal  
jquery.transit     $.transit  
jquery.validate    $.validator  
json2              JSON.stringify  
webfont            WebFont  
jquery.blockUI     $.unblockUI()  
respond            window.respond  
moment             window.moment 

dataTables         $.fn.dataTable

NOTE: List copied from here and the augmented.
Great post on CDN

Quick review of nice Open Source JavaScript Grids / Data Tables

DataTables

Website: http://datatables.net/
This is a very powerful and mature Grid and my top choice. My only complaint is that it is almost overwhelming because it is so feature rich and if you need editing, the coolest features you have to pay for. It supports just about any scenario out there and the docmentation is GREAT. It does a good job making simple cases easy to do though. In fact, they have a generator that generates all the code you need to get started after you specify how you want to use it, but that unfortunately is with the paid Editor. They do have for free a tool to help you figure out exactly what files you need for downloading based on your needs. It does allow cells to have input elements (see example), but there isn't any special support for them unless you buy the Editor. If you go that route, it has inline editing, popup screen editing, and single field editing just like the Editablegrid product (below) does AND is supported specifically with ASP.NET (and PHP), but again only for the paid Editor. It does support having all the rows editable at once though. It is very easy to use, but has lots of ways to do very advanced things. It can get data from the server or client. Works well with large amounts of data. The documentation is very well done. There are lots of examples and they include how it was done. It also has extensions for exporting data to Excel, etc.

Bootstrap Table

Website: http://bootstrap-table.wenzhixin.net.cn/
This one is good and has examples in GitHub, but the documentation / demo on the site is quite light. It does have additional documentation and examples on GitHub though. It is nice that users can export data to multiple formats, choose just the columns they want, search, etc. If you click on a cell it will bring up an editor for that field and can have associated validation with it.

Editablegrid

Website: http://www.editablegrid.net
This one is uniquely designed for inline editing. To edit a cell, just click it and it becomes editable. It supports PHP binding, but doesn't seem to support ASP.NET directly, but may not matter unless you need mass amounts of data editing. This grid is a bit light on the features, but is still pretty nice.

How to change the default Name of a TextBox when using MVC, Razor

We want this format so that each Name on the form is unique. Sometimes we need to change what the value of Name is on the HTML form.

Here is the simple example using @Html.TextBoxFor() to change it from the default of "StartTime" to "MyStartTime":

@Html.TextBoxFor(modelItem => item.StartTime, new { Name = "MyStartTime"})

Below is an example of how to set the Name of the StartTime TextBox when looping through a collection of complex objects such as an Appointment
In the html, the INPUT will have a names like:
Appointments[0].StartTime
Appointments[1].StartTime

The example below formats the Name html property to the format that the MVC binder understands and will automatically create the Appointment objects and bind to them.To better understand how the binding in MVC works, check out this article.

This is possible because @Html.EditorFor takes a parameter for the Name, but @Html.HiddenFor.

If you want to use @Html.TextBoxFor instead, it doesn't have this option, but we can use the htmlAttributes parameter. In this case, Name is CASE SENSITIVE. Using name will not work right.

 @{
        var @i = 0;
    }
    @foreach (var item in Model)
    {
                @Html.EditorFor(modelItem => item.StartTime, "TextBox", "Appointments[" + @i + "].StartTime", null)
                @Html.TextBoxFor(modelItem => item.StartTime, new { Name = "Appointments[" + @i + "].StartTime"})
                 i++;
    }

NOTE: You can also use a for loop instead of the foreach.
NOTE: This technique should also work for the other Html helpers, but I have not tested it.

Get a list of ApplicationUsers (Users) in MVC 6

If you need a list of all the user in your MVC 6 application.

The following will work from any controller.

using Microsoft.AspNet.Identity;
using Microsoft.AspNet.Identity.Owin;
...
 var users = Request.GetOwinContext().GetUserManager().Users;

The generated controllers have a line like this:

private ApplicationDbContext db = new ApplicationDbContext();

After the above initialization takes place, you can also get a list of users by simply doing:

db.Users

Notice, I am NOT using db.ApplicationUsers which you may have been tempted to add to the ApplicationDbContext class using the following:

public DbSet ApplicationUsers { get; set; }

The problem as noted here is that that IdentityDbContext already has the following defined.

public virtual IDbSet Users { get; set; }

So, by adding the ApplicationUsers line above you have added two properties that map to the ApplicationUsers. Here is the error you will get if you have done this:

Multiple object sets per type are not supported. The object sets 'ApplicationUsers' and 'Users' can both contain instances of type 'MyApp.Models.ApplicationUser'.

Command Line to set permissions recursively

Open a command prompt and  run as Administrator. Next execute a command like the following

icacls "D:" /T /grant Everyone:(OI)(CI)F

If you do icacls as the command prompt you will get help on the command.

You can replace Everyone with an approprate user. You can also specify less than full control.

Useful collection of task oriented utilities

NirSoft has an interesting collection of task oriented utilities. Below is a list of them.

• Password Recovery Utilities
  • WebBrowserPassView - View the passwords stored by your Web browser (Supports Internet Explorer, Firefox, Chrome, Safari, and Opera)
  • Mail PassView - Recover the passwords of popular email clients: Outlook Express, MS Outlook, Eudora, Mozilla Thunderbird , and more...
  • MessenPass - Recover the passwords of instant messenger programs: Yahoo Messenger, MSN Messenger, Trillian, and more...
  • Dialupass: Dialup Password Recovery - Recovers the passwords of dialup entries (VPN and Internet connections). Works also under Windows 2000/XP.
  • Network Password Recovery - Freeware utility that recovers the network passwords stored by Windows XP (Credentials file).
  • BulletsPassView - Reveal the passwords hidden behind asterisk ('****') or bullets characters in standard password text-boxes.
  • SniffPass - Password Sniffer - Listen to your network, and capture POP3, IMAP4, SMTP, FTP, and HTTP (basic authentication) passwords
  • PstPassword - Recover the password of Outlook PST file.
• Network Monitoring Tools
  • WirelessNetView - View the details of all wireless network in your area (SSID, Signal Quality, MAC Address, and more...)
  • Wireless Network Watcher - Show who is connected to your wireless network.
  • BluetoothView - Monitor the Bluetooth activity around you.
  • SmartSniff: TCP/IP Sniffer - Capture TCP/IP packets on your network adapter and view the captured data as sequence of conversations between clients and servers.
  • CurrPorts: TCP/IP Connections Viewer - Freeware tool that displays the list of all currently opened TCP and UDP ports on your local computer.
  • AdapterWatch - displays useful information about your network adapters: IP addresses, Hardware address, WINS servers, DNS servers, MTU value, Number of bytes received or sent, The current transfer speed, and more...
  • DownTester - Test the download speed of your Internet connection.
• Internet Related Utilities
  • IPNetInfo - Find all available information about an IP address: The owner of the IP address, the country/state name, IP addresses range, contact information (address, phone, fax, and email), and more.
  • MyLastSearch - View your latest searches with Google, Yahoo, and MSN.
  • VideoCacheView - Copy video files (.flv and others) from the Web browser cache and temporary folder.
  • WebVideoCap - Capture Flash video (.flv) and RTSP/MMS streams from Video Web sites.
  • SiteShoter - Take a screenshot of a Web site.
  • DNSDataView - View the DNS records of specified domains.
  • IECookiesView: Cookies Viewer/Manager for IE - View/Delete/Modify the cookies that Internet Explorer stores on your computer. Read More >>
  • IEHistoryView - View/Delete the URLs that you visited in the last few days.
  • HTMLAsText (HTML To Text Converter) - Freeware utility that converts HTML files to text files.
  • WhoisThisDomain - Get information about a registered domain.
  • DNSDataView - DNS lookup tool.
• MS-Outlook Tools
  • NK2Edit - Edit, delete, add, merge, and repair records in the AutoComplete files (.NK2) of Outlook.
  • OutlookAttachView - View and extract attachments in your Outlook mailbox.
  • OutlookStatView - Get statistics about your Outlook mailbox.
  • OutlookAddressBookView - View / export the address book of MS-Outlook.
• Command-Line Utilities
  • NirCmd: Freeware Command-Line Tool - Do some useful tasks from command-line: turn off your monitor, turn off the computer, open/close the door of your cd-rom drive, dial to VPN/Internet connection, change your display settings, and much more !!
• Desktop Utilities
  • SearchMyFiles - Alternative to 'Search For Files And Folders' module of Windows and also duplicate files search.
  • MyUninstaller - Alternative to the standard Add/Remove applet of Windows operating system.
  • Volumouse - Control the sound volume with your wheel mouse !!
  • FileTypesMan - Alternative to 'File Types' manager of Windows.
• Freeware System Tools
  • ProduKey - Recover Office/Windows CD-Key
  • ShellExView - Displays the details of shell extensions installed on your computer, and allows you to easily disable and enable each shell extension.
  • WhatInStartup - Add, delete, modify or disable programs that run at Windows startup.
  • USBDeview - View all installed/connected USB devices on your system.
  • DevManView - Alternative to device manager of Windows.
  • DriverView - List all device drivers currently loaded on your Windows.
  • RegScanner (Registry Scanner) - Scan and find values in the Registry.
  • OpenedFilesView - View opened/locked files in your system.
  • SysExporter - Grab the data stored in standard list-views, list boxes, and combo boxes from almost any application running on your system, and export it to text, HTML or XML file.
  • BlueScreenView - View crash information stored in the MiniDump files created on blue screen of Windows.
   

Open Socket using PowerShell

To open a socket connection on a specific host and port using PowerShell do the following:

(new-object Net.Sockets.TcpClient).Connect("proxy.us.dhl.com", "8080")

If it can't connect it will timeout. If it can connect it will return to the Powershell prompt.

wget equivalent in PowerShell

As some have pointed out there is not the equivalent for wget in PowerShell 1 or 2. It is included as Invoke-WebRequest in PowerShell 3.

To implement it yourself in PowerShell 1 or 2, you can use the following:



## Declare a function that takes source and destination arguments
Function Get-Webclient ($url, $out) {
$proxy = [System.Net.WebRequest]::GetSystemWebProxy()
$proxy.Credentials = [System.Net.CredentialCache]::DefaultCredentials
$request = New-Object System.Net.WebCLient
$request.UseDefaultCredentials = $true ## Proxy credentials only
$request.Proxy.Credentials = $request.Credentials
$request.DownloadFile($url, $out)
}

## Call the function with our source and destination.
Get-Webclient "http://www.google.com" "C:\Foo3.txt"

Credit to this post.

Merge multiple MP3 files into one file

It is not a simple thing to merge MP3 files because they have header information at the beginning of each file before the actual content. So, in theory you can't just merge them together without writing a single header for the new merged file. However, many MP3 players will still read the files okay even though there are multiple headers (all but one mixed where the content should be). This hack exploits this robust implementation of MP3 players.

WARNING: This is most definitely a hack and does not always work because it is technically not correct. However, in a pinch it can work (sometimes). Do not trash your original files unless you don't mind losing the data.

Here is the very simple DOS command for merging multiple MP3 files into one MP3 file (that will likely play on most MP3 players, but is technically not correctly formatted so it will not play on all MP3 players).

copy /b *.mp3 output.mp3

Using Powershell to email free space

Below are two script


Put the following into a PowerShell file (EmailFreeSpace.ps1) and execute.

Clear-Host
$Body = Get-WmiObject Win32_logicaldisk -ComputerName LocalHost `
| Format-Table DeviceID, MediaType, `
@{Name="Size(GB)";Expression={[decimal]("{0:N0}" -f($_.size/1gb))}}, `
@{Name="Free Space(GB)";Expression={[decimal]("{0:N0}" -f($_.freespace/1gb))}}, `
@{Name="Free (%)";Expression={"{0,6:P0}" -f(($_.freespace/1gb) / ($_.size/1gb))}} `
-AutoSize


$EmailFrom = "test@gmail.com"
$EmailTo = "test@gmail.com"
$Subject = "Notification from XYZ"  
$SMTPServer = "smtp.gmail.com"
$SMTPClient = New-Object Net.Mail.SmtpClient($SmtpServer, 587)
$SMTPClient.EnableSsl = $true
$SMTPClient.Credentials = New-Object System.Net.NetworkCredential("test@gmail.com", "password here");
$SMTPClient.Send($EmailFrom, $EmailTo, $Subject, $Body)

The output of the first part of the script will look like this:

DeviceID MediaType Size(GB) Free Space(GB) Free (%)
-------- --------- -------- -------------- --------
C:              12      232            174   75 %
D:              12      233            155   67 %
H:               0     7500           5634   75 %
J:               0      100            100  100 %
K:              11        0              0
Q:              12        0              0

The second part of the script will send an email using Gmail. You need to have a Gmail account and you will need to put the password on the second to last line of the script. You can also use any other STMP server such as Hotmail, etc.

Understanding GitHub

GitHub is a great improvement over subversion if for now other reason it has great performance for global teams that have network latency. It can be confusing when learning it though.To make things easier GitHub has created a GUI called GitHub for Windows, so you don't have to use the command line. The command line can do anything GitHub for Windows can. Here is one of the best tutorials I have found for using the GUI. It can still be a bit confusing because of the 2-3 repositories suggested by GitHub. Below is a diagram to show the steps needed to make a change, share it with others, and merge it back into the main repository.

Let's walk through the diagram in detail. The first thing to notice in this example is that there is an existing repository called AwesomeCode in github.com in the account for user Jane. The goal is that all changes will end up back in Jane's repository (repo). If your goal is not to contribute to the main repository and just want to work on your own copy (aka fork) that will not affect the main repo at all you don't create pull requests. The steps below assume you would like to contribute to the main repo (Jane's).

1. Fork - A fork is like a branch or a copy of the main repo. To fork a repo you need to use github.com and click the fork button. This will not do anything to your laptop. It will copy the main repo to your github.com account. Notice the difference in the urls for the main and your forked copy.

2. Clone - When you clone a repo you are getting a local copy of it on your laptop. You can use github.com to do this or you can use GitHub for Windows. When you do this you will need to specify the directory where the repo will be located. This command will create files in the specified directory, but it will also create a .git directory in that same directory you specified. This is known as the Local Repository or Index. The name index because there is and Index file in the .git directory.

3. Modify File - Here is where you use your favorite editor or program to modify or create a file.

4. Save - This is simply using your favorite editor or program to save the changes you made to the file to the local disk. If you re-open the file you will see the changes.

5. Commit - Commit in GitHub is different than in Subversion, CVS, etc because you don't have to be connected to the internet, cloud, network, etc to commit the change. All your commits will be done to your Local Repo. This make working with GibHub very efficient for networks with latency. Commits are not visible on the main or even your repo on github.com yet. If your laptop was lost so would your changes.

6. Push / Sync - Push is what the command is called, but in GitHub for Windows it is the Sync button. When you Push / Sync the changes on your Local Repo are pushed (similar to a commit) to your repository on github.com. You changes are now visible on your repository on github.com, but it is NOT visible in the main repository (Jane's). If you don't want to make your changes available to Jane then you are done. You have updated your repository.

7. Create Pull Request - A pull request is a way to say to share what you have done with others. You can use the pull request to get approval for your change, or get feedback, or help, etc. Other users can then comment on your pull request. It is the last step before merging your changes with the main repo. A pull request is a made while on your repository on github.com or you can now use GitHub for Windows to do it, but the pull request is actually available on the main repo (Jane's) under Pull Requests. Try to keep your pull requests small (5-10 lines or changes) and descriptive so other people can understand what your were doing.

8. Merge Pull Request - When you want to incorporate a the changes in a Pull Request into the main repo you need to merge the changes. Sometimes it can be automatically done, but other times it can't. Once the changes are merged the changes you made in your Pull Request are now part of the main repo. The general rule is that no Pull Requests should be merged into the main repo unless they are ready for releasing to production.

Sysinternals is back!

If you were a fan of the Sysinternals tools by Mark Russinovich, then you will be happy to know that they are back. Thankfully they moved to Microsoft (a bit later) with Mark.

Here is the link to the new Sysinternals Suite.

Here are links to the individual utilities:

AccessChk AccessEnum AdExplorer AdInsight AdRestore Autologon Autoruns BgInfo CacheSet ClockRes Contig Coreinfo Ctrl2Cap DebugView Desktops Disk2vhd DiskExt DiskMon DiskView Disk Usage (DU) EFSDump FindLinks Handle

Hex2dec Junction LDMDump ListDLLs LiveKd LoadOrder LogonSessions MoveFile NTFSInfo PendMoves PipeList PortMon ProcDump Process Explorer Process Monitor PsExec PsFile PsGetSid PsInfo PsPing PsKill PsList PsLoggedOn

PsLogList PsPasswd PsService PsShutdown PsSuspend RAMMap RegDelNull Registry Usage (RU) RegJump SDelete ShareEnum ShellRunas Sigcheck Streams Strings Sync Sysmon TCPView VMMap VolumeID WhoIs WinObj ZoomIt

Thank you Mark!

Make sure my Windows Service is enabled and running

The situation

I recently had a need to keep the ASP.NET State Service (aspnet_state) running. For some reason it was getting disabled and stopped sometimes. I assume a corporate security policy is doing it periodically, but I don't know exactly how or why it happens. However, I do know that every time the ASP.NET State Service goes down so does my web applications on my web farm because they are using the ASP.NET State Service.

The Workaround

Luckily, PowerShell makes it very easy to set the StartupType to Automatic and then start a Windows Service. No need to test state before calling since it seems to only change something if it is needed.

Set-Service -name aspnet_state -StartupType Automatic

Set-Service -name aspnet_state -Status Running

Save the two above lines in a file with any name you like ending in .ps1. I'll use EnsureStateService.ps1 for this example.

Automating the workaround

Every minute I want to execute the above PowerShell file. That way at most I will have 1 minute of downtime (minus the time it takes to enable and start the ASP.NET State Service). To create a scheduled task in Windows Task Scheduler do the following:

1. Create a task
2. Set the task to be executed as the user System.
3. I used Windows Vista, Windows Server 2008 configuration, but I don't think it matters in this case.
4. For the trigger, I set it to run Daily, running every 1 days, Repeated task every 1 minute, and stopped task if running for more than 1 minute as shown below:

5. For the Action, do the following:

For Program/script field enter PowerShell

For Add argument (optional): field enter the path to the powershell script file surrounded by double-quotes. In this example, it is "D:\MDM\EnsureStateService.ps1"

Scrum Activities Overview

Sprint Planning Meeting

First Half

Duration: 2 hours (for a 2 weeks spring), but should be scaled to the length of the sprint. For example, 4 hours for a 4 week sprint.
Frequency: Before starting a sprint
Participants:

• Scrum Master
• Development Team
• Product Owner
• Steakholders

Deliverables:

• Product Backlog (user stories) that is prioritized and estimated in terms of hours or difficulty.

Description: Agree on what should be in the sprint

Second Half

Duration: 2 hours (for a 2 weeks spring), but should be scaled to the length of the sprint. For example, 4 hours for a 4 week sprint.
Frequency: Before starting a sprint
Participants:

• Scrum Master
• Development Team
• Product Owner
• Steakholders

Deliverables:

• Sprint Backlog that contains development tasks that will fit that the development team thinks can be completed during the sprint
• Plot ideal burn-down chart

Description: Hashout plan for sprint

Daily Scrum Meeting (aka Stand-up meeting)

Duration: 15 minutes

Frequency: Daily

Participants:

• Scrum Master
• Development Team
• Product Owner
• Steakholders

Deliverables

• Impediments (recorded by Scrum Master).
  NOTE: Not resolved during this meeting.

Description:

The scrum master asks each person on the team three questions:

• What did you accomplish yesterday?
• What are you working on today?
• What impediments are in your way?

Daily Updates

Update Sprint Burndown Chart: At the end of each day each developer updates the number of hours remaining for the task s/he is working on.

Sprint Review Meeting

Duration: Max of four hours

Frequency: At the end of the sprint

Participants:

• Scrum Master
• Development Team
• Product Owner
• Steakholders

Deliverables

• Feedback from product owner, steakholders

Description:

Demonstration of what was implemented in the spring

Sprint Retrospective Meeting

Duration: Max of 3 hours

Frequency: At the end of the sprint

Participants:

• Scrum Master
• Development Team
• Product Owner
• Steakholders

Deliverables

• Document:
• What went well during the sprint?
• What could be improved in the next sprint
• Improve process as needed

Description:

Reflect on the sprint.

References:

• Getting Agile Part II Introducing Scrum
• Scrum (Wikipedia)

Grant Permissions to create, alter, or view definition of a view in SQL Server

Let's assume you have a SQL Server user called reporting_user and it has select only access to some tables. To give that user the ability to create views use the following statements (be sure to change reporting_user to the name of your user).

grant create view to reporting_user
grant select on schema :: dbo to reporting_user
grant view definition on schema :: dbo to reporting_user

HBase Basics

What is Hadoop

HBase is the Hadoop database modeled after Google's Bigtable. It is an Apache Top Level Project. This means it is open source. It is however embraced and supported by IBM, etc. It is used by industry heavy hitters like Facebook, Twitter, etc to access BigData. It is written in Java, but there are other API to access it. It has the following characteristics:

• Sparse - data is scattered
• Distributed - spread out over commodity hardware
• Persistent - data will be saved
• Multi-dimensional - may be multiple versions of data
• Sorted Map - need a key to access the data

NoSQL Technology

• HBase is a NoSQL datastore
• NoSQL stands for "Not only SQL"
• Not intended to replace a RDBMS
• Suited for specific business needs that require
• Massive scaling to terabytes and petabytes and larger
• Commodity Hardware used for scaling out solution
• Not knowing schema upfront

Why HBase

• HBase CAN replace costly implementations of RDBMS for BigData applications, but is not meant to replace RDBMS entirely because 
• It doesn't support SQL
• Not for transactional processing
• Does not support table joins
• Horizontal scaling for very large data sets
• Ability to add commodity hardware without interruption of service
• Don't know data types in advance. This allows for a flexible schema.
• Need RANDOM read/write access to BigData. Reads and writes are very quick and efficient.
• Sharding - sharing the data between nodes

NOTE: Everything is stored as an array of bytes (except timestamp which is stored as a long integer).

HBase vs. RDBMS

Topic	HBase	RDBMS
Hardware architecture	Similar to Hadoop. Clustered commodity hardware. Very affordable.	Typically large scalable multi-processor systems. Very expensive.
Typical Database Size	Terabytes to Petabytes - hundreds of millions to billions of rows	Gigabytes to Terabytes - hundreds of thousands to millions of rows.
Data Layout	A sparse, distributed, persistent, multi-dimensional, sorted map.	Rows or column oriented
Data Types	Bytes only	Rich data type support
Transactions	ACID support on a single row only	Full ACID compliance across rows and tables
Query Language	API primitive commands only, unless combined with Hive or other technologies.	SQL
Indexes	Row-Key only unless combined with other technologies such as Hive or IBM's BigSQL	Yes. On one or more columns.
Throughput	Millions of queries per second	Thousands of queries per second
Fault Tolerance	Built into the architecture. Lots of nodes means each is relatively insignificant. No need to worry about individual nodes.	Requires configuration of the HW and the RDBMS with the appropriate high availability options.

Data Representation Example (RDBMS vs HBase)

RDBMS might look something like this

ID (Primary Key)	LName	FName	Password	Timestamp
1234	Smith	John	Hello, world!	20130710
5678	Doe	Jane	wysiwyg	20120825
5678	Doe	Jane	wisiwig	20130916

Logical View in HBase

Row-Key	Value (Column-Family, Qualifier, Version)
1234	info {'lName': 'Smith', 'fName': 'John' } pwd {'password': 'Hello, world!' }
5678	info {'lName': 'Doe', 'fName': 'Jane' } pwd {'password': 'wysiwyg'@ts 20130916, 'password': 'wisiwig'@ts 20120825 }

HBase Physical (How it is stored on disk)

Logical View to Physical View

Let's assume you want to read Row4. You will need data from the both physical files. In the case of CF1, you will get two rows since there are two versions of the data.

HBase Components

Region

• This is where the rows of a table are stored
• Each region stores a single column family
• A table's data is automatically sharded across multiple regions when the data gets too large.

Region Server

• Contains one or more regions
• Hosts the tables, performs reads and writes, buffers, etc
• Client talks directly to the Region Server for their data.

Master

• Coordinating the Region Servers
• Detects status of load rebalancing of the Region Servers
• Assigns Regions to Region Servers
• Multiple Masters are allowed, but only one is the true master, and the others are only backups.
• Not part of the read/write path
• Highly available with ZooKeeper

ZooKeeper

• Critical component for HBase
• Ensures one Master is running
• Registers Region and Region server
• Integral part of the fault tolerance on HBase

HDFS

• The Hadoop file system is where the data (physical files) are kept

API

• The Java client API.
• You can also use SQL is you use Hive to access your data.

Here is how the components relate to each other.

HBase Shell introduction

Starting HBase Instance
HBASE_HOME/bin/start-hbase.sh

Stopping HBase Instance
HBASE_HOME/bin/stop-hbase.sh

Start HBase shell
HBASE_HOME/bin/hbase shell

HBase Shell Commands

See a list of the tables

list

Create a table

create 'testTable', 'cf'

NOTE: testTable is the name of the table and cf is the name of the column family

Insert data into a table
Insert at rowA, column "cf:columnName" with a value of "val1"
put 'testTable', 'rowA', 'cf:columnName', 'val1'

Retrieve data from a table
Retrieve"rowA"from the table "testTable"
get 'testTable', 'rowA'

Delete data from a table
delete 'testTable', 'rowA', 'cf:columnName', ts1.

Delete a table:
disable 'testTable'
drop 'testTable'

HBase Clients

HBase Shell - you can do the above crud operations using the HBase Shell. However will be limiting for more complicated tasks.
Java - you can do the above crud operations and more using Java. It will be executed as a MapReduce job.

NOTE: Some of this material was copied directly from the BigData University online class Using HBase for Real-time Access to your Big Data - Version 2.If you want hands on labs, more explanation, etc I suggest you check it out since all the information on this post comes from there.